-- Logs begin at Fri 2022-10-21 20:43:40 UTC, end at Fri 2022-10-21 22:09:23 UTC. -- Oct 21 21:07:07.459856 dv-cinder-861566-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 21 21:07:07.467629 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 21 21:07:07.469609 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Oct 21 21:07:07 2022] *** Oct 21 21:07:07.469609 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 21 21:07:07.469609 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 21 21:07:07.469609 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: nodename: dv-cinder-861566-4-iscsi Oct 21 21:07:07.469609 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: machine: x86_64 Oct 21 21:07:07.469609 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: clock source: unix Oct 21 21:07:07.469609 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: pcre jit disabled Oct 21 21:07:07.470045 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: detected number of CPU cores: 4 Oct 21 21:07:07.470045 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: current working directory: / Oct 21 21:07:07.470045 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: detected binary path: /usr/bin/uwsgi-core Oct 21 21:07:07.470045 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: your processes number limit is 47680 Oct 21 21:07:07.470045 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: your memory page size is 4096 bytes Oct 21 21:07:07.470045 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: detected max file descriptor number: 2048 Oct 21 21:07:07.470045 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: lock engine: pthread robust mutexes Oct 21 21:07:07.470045 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: thunder lock: enabled Oct 21 21:07:07.470045 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 21 21:07:07.470045 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 21 21:07:07.510507 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: Python main interpreter initialized at 0x5600c3d569c0 Oct 21 21:07:07.510507 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: python threads support enabled Oct 21 21:07:07.510507 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: your server socket listen backlog is limited to 100 connections Oct 21 21:07:07.510507 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: your mercy for graceful operations on workers is 90 seconds Oct 21 21:07:07.510874 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: mapped 403077 bytes (393 KB) for 2 cores Oct 21 21:07:07.510963 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: *** Operational MODE: preforking *** Oct 21 21:07:07.511224 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: *** uWSGI is running in multiple interpreter mode *** Oct 21 21:07:07.511224 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: spawned uWSGI master process (pid: 81291) Oct 21 21:07:07.511421 dv-cinder-861566-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 21 21:07:07.515609 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: spawned uWSGI worker 1 (pid: 81305, cores: 1) Oct 21 21:07:07.515609 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: spawned uWSGI worker 2 (pid: 81306, cores: 1) Oct 21 21:07:07.515609 dv-cinder-861566-4-iscsi devstack@keystone.service[81291]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 21 21:07:11.777506 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 21 21:07:11.777506 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: from cryptography.utils import int_from_bytes Oct 21 21:07:11.778165 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 21 21:07:11.778165 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: from cryptography.utils import int_from_bytes Oct 21 21:07:11.778282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 21 21:07:11.778282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: from cryptography.utils import int_from_bytes Oct 21 21:07:11.778282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 21 21:07:11.778282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: from cryptography.utils import int_from_bytes Oct 21 21:07:11.828530 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 21 21:07:11.828728 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 21 21:07:11.828834 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 21 21:07:11.828923 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 21 21:07:11.829002 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] command line args: [] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 21 21:07:11.829079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] command line args: [] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 21 21:07:11.829160 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 21 21:07:11.829344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ================================================================================ {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 21 21:07:11.829424 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 21 21:07:11.829521 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ================================================================================ {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 21 21:07:11.829599 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] admin_token = **** {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.829770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] admin_token = **** {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.829858 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] config_dir = [] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.829996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.830074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] config_dir = [] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.830179 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] config_source = [] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.830265 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.830343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] config_source = [] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.830424 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] debug = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.830523 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] debug = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.830609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.830849 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.830995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.831100 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.831196 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.831296 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.831391 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.831490 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.831651 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.831763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.831862 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] list_limit = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.831962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] list_limit = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.832057 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] log_config_append = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.832163 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] log_config_append = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.832307 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.832416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.832518 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] log_dir = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.832625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] log_dir = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.832720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] log_file = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.832814 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] log_file = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.832940 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.833021 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.833145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.833222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.833336 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.833417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.833500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.833620 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.833702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.833781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.833904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.834032 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.834120 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.834199 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.834324 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.834406 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.834532 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.834616 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.834689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.834803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.834878 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.834993 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.835082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.835164 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.835236 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.835393 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.835474 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.835660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.835756 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.835932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.836198 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.836381 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] publish_errors = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.836458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.836586 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] publish_errors = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.836667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.836811 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.836895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.837021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.837099 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.837195 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.837272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.837438 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.837528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.837666 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.837750 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] standard_threads = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.837894 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] standard_threads = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.837979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.838118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.838198 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.838305 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.838381 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.838480 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] use_journal = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.838565 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.838685 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] use_json = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.838765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] use_journal = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.838836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] use_stderr = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.838912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] use_json = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.838983 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] use_syslog = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.839052 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] use_stderr = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.839151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] use_syslog = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.839230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.839375 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 21:07:11.839453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.839651 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.839734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.839889 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.839970 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.840144 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.840230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.840378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.840460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.840616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.840696 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.840847 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.840925 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.841084 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.841162 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.841313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.841391 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.841555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.841633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.841798 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.841876 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.842028 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.842106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.842262 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.842348 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.842750 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.842847 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.842847 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.843000 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.843072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.843148 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.843220 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.843345 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.843433 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.843507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.843675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.843813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.843932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.844116 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.844200 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.844271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.844394 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.844473 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.844644 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.844731 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.844894 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.844984 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.845142 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.845227 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.845373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.845463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.845603 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.845687 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.845829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.845906 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.846052 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.846140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.846283 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.846362 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.846528 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.846608 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.846755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.846841 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.846982 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.847060 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.847205 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.847282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.847448 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.847535 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.847702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.847781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.847935 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.848023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.848167 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.848252 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.848396 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.848495 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.848627 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.848716 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.848853 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.848943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.849078 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.849157 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.849302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.849378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.849525 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.849676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.849751 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.849828 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.849968 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.850047 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.850207 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.850293 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.850431 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.850514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.850655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.850733 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.850878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.850964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.851109 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.851185 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.851478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.851624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.851624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.851825 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.851924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.852019 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.852113 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.852724 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.852836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.852954 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.853029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.853144 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.853252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.853378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.853463 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.853580 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.853655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.853770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.853884 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.853964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.854120 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.854241 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.854350 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.854438 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.854576 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.854668 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.854830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.854911 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.855062 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.855182 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.855290 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.855370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.855522 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.855667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.855785 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.855866 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.856036 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.856168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.856301 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.856431 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.856575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.856694 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.856817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.856945 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.857069 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.857190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.857321 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.857442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.857576 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.857699 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.857830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.857956 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.858085 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.858212 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.858347 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.858471 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.858594 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.858723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.858854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.858981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.859113 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.859238 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.859363 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.859487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.859645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.859788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.859911 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.860038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.860170 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.860296 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.860296 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.860541 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.860541 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.860777 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.860902 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.861043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.861180 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.861306 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.861430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.861553 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.861553 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.861803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.861803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.862118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.862118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.862357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.862357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.862601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.862732 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.862869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.862996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.863119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.863243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.863369 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.863495 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.863670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.863794 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.863920 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.864045 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.864168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.864296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.864455 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.864578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.864711 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.864834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.864962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.865089 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.865223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.865347 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.865492 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.865617 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.865743 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.865895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] auth.external = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.865895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.866147 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.866147 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] auth.external = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.866393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.866393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.866633 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.866757 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] auth.password = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.866882 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.867017 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] auth.token = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.867152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.867277 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.867402 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] auth.password = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.867527 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.867667 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] auth.token = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.867837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.868495 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.868625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.868761 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.868889 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.869016 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.869154 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.869281 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.869408 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.869545 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.869674 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] credential.caching = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.869806 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.869942 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.870068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.870200 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.870327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] credential.caching = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.870454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.870580 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.870716 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.870842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.870968 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.871099 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.871235 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.871361 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.871496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.871666 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.871790 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.871927 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.872053 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.872179 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.872314 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.872447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.872447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.872688 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.872820 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.872947 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.872947 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.873207 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.873332 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.873462 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.873587 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.873720 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.873849 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] federation.caching = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.873980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.874106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.874233 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.874315 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.874488 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] federation.caching = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.874615 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.874743 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.874881 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.875007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.875133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.875267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.875392 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.875518 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.875658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.875786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.875911 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.876049 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.876177 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.876304 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.876431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.876572 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.876700 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.876835 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.876963 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] identity.caching = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.877089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.877089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.877217 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.877288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] identity.caching = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.877436 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.877436 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.877576 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.877650 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.877722 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.877799 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.877880 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.877955 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.878026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.878098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.878098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.878224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.878224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.878362 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.878434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.878505 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.878505 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.878632 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.878714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.878795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.878875 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.878948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.879026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.879099 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.879099 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.879248 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.879325 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.879409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.879482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.879554 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.879698 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.879773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.879845 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.879916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.879986 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.880058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.880058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.880197 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.880269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.880340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.880411 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.880482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.880482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.880669 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.880753 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.880826 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.880898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.880978 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.881047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.881118 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.881190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.881261 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.881331 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.881410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.881484 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.881604 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.881721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.881837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.881969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.882083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.882219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.882329 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.882451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.882560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.882695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.882783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.882939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.883036 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.883166 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.883247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.883436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.883525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.883689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.883856 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.883937 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.884097 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.884176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.884333 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.884460 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.884582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.884663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.884812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.884897 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.885046 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.885126 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.885277 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.885369 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.885490 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.885566 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.885719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.885801 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.885963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.886044 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.886199 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.886438 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.886568 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.886668 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.886780 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.886905 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.887059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.887160 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.887296 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.887378 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.887525 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.887626 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.887788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.887869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.888043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.888162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.888278 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.888355 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.888505 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.888586 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.888750 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.888834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.888978 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.889058 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.889209 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.889290 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.889434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.889515 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.889664 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.889745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.889894 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.889974 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.890126 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.890207 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.890350 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.890431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.890594 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.890681 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.890820 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.890900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.891063 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.891143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.891538 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.891688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.891802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.891926 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.892025 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.892144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.892255 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.892336 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.892490 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.892571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.892720 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.892800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.892949 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.893029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.893176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.893257 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.893413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.893493 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.893653 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.893734 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.893882 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.893962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.894111 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.894232 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.894344 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.894423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.894582 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.894666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.894811 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.894892 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.895037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.895116 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.895265 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.895344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.895500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] resource.caching = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.895606 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.895760 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.895840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.895988 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.896069 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.896219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.896300 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.896456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] resource.caching = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.896544 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.896699 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.896781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.896929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.897009 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.897163 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.897258 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.897394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.897473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.897629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.897710 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.897861 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.898014 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] role.caching = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.898092 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] role.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.898162 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.898317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.898397 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.898548 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.898635 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.898778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] role.caching = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.898859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.899041 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] role.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.899121 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.899268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.899357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.899512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.899613 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.899778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.899858 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.900021 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.900114 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.900263 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.900343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.900494 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.900575 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.900734 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.900821 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.900976 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.901056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.901216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.901295 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.901443 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.901532 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.901685 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.901837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.901917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.902071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.902152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.902319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.902400 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.902567 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.902760 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.902841 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.902992 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.903074 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.903237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.903318 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.903482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.903562 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.903749 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.903832 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.903991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.904070 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.904217 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.904296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.904443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.904523 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.904672 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.904756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.904919 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.904997 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.905146 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.905228 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.905372 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.905452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.905599 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.905679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.905843 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.905924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.906071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.906150 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.906309 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] token.caching = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.906389 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.906543 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.906626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.906773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.906852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.907002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.907082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.907233 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] token.caching = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.907314 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.907464 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.907544 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.907729 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.907810 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.907958 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.908039 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.908200 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.908280 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.908440 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.908520 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.908671 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.908752 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.908907 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.908986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.909135 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.909213 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.909374 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.909455 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.909605 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.909689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.909837 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.909917 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.910066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.910159 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.910307 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.910387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.910576 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.connection = **** {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.910728 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.910808 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.910962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.911052 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.911200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.911282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.911461 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.911542 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.911729 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.911811 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.connection = **** {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.911962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.912045 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.912193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.912274 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.912425 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.912504 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.912661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.912742 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.912889 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.912968 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.913117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.913197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.913347 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.913428 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.913575 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.913654 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.913812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.913892 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.914043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.914124 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.914266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.914345 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.914498 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.914577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.915238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.915321 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.915436 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 21 21:07:11.915516 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.915743 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.915952 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.916187 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.916417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 21:07:11.916583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 21 21:07:11.944903 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:11.945129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:11.945232 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:11.945352 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:11.945438 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:11.945571 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:11.953767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:11.953965 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:11.982476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:11.982702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:11.983390 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:11.983648 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.035632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.036059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.083724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.083956 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.084173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.084404 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.236317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.236573 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.236667 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.236747 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.236820 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.236964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.237049 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.237182 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.237366 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.237451 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.237601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.237812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.260157 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.260391 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.260578 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.260677 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.260811 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.260895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.261029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.261111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.261246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.261344 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.261461 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.261543 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.261672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.261877 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.272747 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.273106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.285991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.286181 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.286366 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.286459 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.286597 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.286680 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.286817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.286898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.287029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.287121 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.287242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.287326 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.287453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.287686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 21:07:12.303859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.304070 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 21 21:07:12.304414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.304613 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 21 21:07:12.304887 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.304984 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.305091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 21 21:07:12.305174 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.306001 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.306163 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.306247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 21 21:07:12.306431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.306514 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.306702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.306961 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.307216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.315766 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.316049 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.316323 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.316607 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.316882 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.317159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.317428 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.317706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.324104 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.325196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.326973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.327258 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.327545 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.327855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.328082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.328220 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.328360 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.328488 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.328614 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.328739 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.328897 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.329026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.329161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.329287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.329414 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.329550 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.329676 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.329809 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.329938 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.330186 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.330457 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.330721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.352649 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.354104 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.354387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.358395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.360310 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.360596 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.360882 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.361170 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.361454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.365214 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.365497 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.365775 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.367816 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.368613 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.372680 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.372822 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.372979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.373117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.373272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.373557 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.373834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.374113 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.374397 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.374678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.378602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.378897 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.379181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.379467 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.379773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.380049 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.380318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.380581 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.390263 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.390543 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.394665 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.396320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.396595 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.397870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.398180 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.400517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.402416 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.403739 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.404034 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.405555 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.408204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.408792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.409095 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.409396 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.409695 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.409986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.410274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.411320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.415077 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.415557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.416120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.416455 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.416820 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.417222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.427651 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.427953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.428239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.428484 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.436460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.438204 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.440045 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.440344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.443320 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.443424 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.443502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.443799 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.443938 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.444081 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.444220 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.444483 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.444640 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.454607 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.456906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.457079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.457330 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.457646 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.457924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.458262 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.458654 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.458883 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.459443 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.459945 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.464262 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.464703 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.465027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.465450 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.466107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.475137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.478555 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.479081 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.479384 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.479675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.479971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.480359 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.480639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.480919 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.481435 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.482867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.484491 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.484787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.485083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.491372 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.493414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.496433 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.496697 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.496785 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.496994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.497273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.499551 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.502432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.503857 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.505012 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.505307 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.505391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.505588 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.505670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.505987 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.512300 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.513467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.513755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.513836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.514041 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.514321 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.514598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.514882 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.515156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.515479 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.515792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.516065 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.516596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.516848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.516930 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.517134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.517224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.517396 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.517510 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.517785 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.525525 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.525828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.526092 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.532927 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 21:07:12.533211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.533490 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.533765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.534036 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.534307 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.534577 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.534890 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.535178 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.535457 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.535758 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.536076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.536293 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.536550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 21:07:12.550677 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5600c3d569c0 pid: 81305 (default app) Oct 21 21:07:12.553946 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4e838-d21b-49b4-829f-c8b749d50517 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:12.554236 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4e838-d21b-49b4-829f-c8b749d50517 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:12.554509 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4e838-d21b-49b4-829f-c8b749d50517 None None] PATH_INFO: `/v3` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:12.559200 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Fri Oct 21 21:07:12 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:12.571240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5600c3d569c0 pid: 81306 (default app) Oct 21 21:07:19.316364 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04c87d-a7d0-49ca-bbb4-1511068559a7 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:19.316888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04c87d-a7d0-49ca-bbb4-1511068559a7 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:19.316988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04c87d-a7d0-49ca-bbb4-1511068559a7 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:19.318274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 2/2] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:07:19.326009 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b26ba4-46a9-46f3-bcb0-49fb38d26719 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:19.326577 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b26ba4-46a9-46f3-bcb0-49fb38d26719 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:19.326577 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b26ba4-46a9-46f3-bcb0-49fb38d26719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:19.397169 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: INFO dbcounter [None req-67b26ba4-46a9-46f3-bcb0-49fb38d26719 None None] Registered counter for database keystone Oct 21 21:07:19.416661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG oslo_db.sqlalchemy.engines [None req-67b26ba4-46a9-46f3-bcb0-49fb38d26719 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81306) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 21 21:07:19.423188 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writer thread running {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 21 21:07:19.597844 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG passlib.handlers.bcrypt [None req-67b26ba4-46a9-46f3-bcb0-49fb38d26719 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81306) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 21 21:07:19.598146 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG passlib.handlers.bcrypt [None req-67b26ba4-46a9-46f3-bcb0-49fb38d26719 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81306) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 21 21:07:19.655532 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-67b26ba4-46a9-46f3-bcb0-49fb38d26719 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:19.657123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-67b26ba4-46a9-46f3-bcb0-49fb38d26719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:19.783514 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1/3] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 21 21:07:20.293501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04c87d-a7d0-49ca-bbb4-1511068559a7 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:20.363052 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: INFO dbcounter [None req-6a04c87d-a7d0-49ca-bbb4-1511068559a7 None None] Registered counter for database keystone Oct 21 21:07:20.383999 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG oslo_db.sqlalchemy.engines [None req-6a04c87d-a7d0-49ca-bbb4-1511068559a7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81305) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 21 21:07:20.390099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writer thread running {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 21 21:07:20.509308 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04c87d-a7d0-49ca-bbb4-1511068559a7 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:20.513554 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a14b9-560f-4e33-b057-9fde902dd485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:20.514204 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a14b9-560f-4e33-b057-9fde902dd485 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:20.514487 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a14b9-560f-4e33-b057-9fde902dd485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:20.514759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a14b9-560f-4e33-b057-9fde902dd485 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:20.523097 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a14b9-560f-4e33-b057-9fde902dd485 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:20.575232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e50a14b9-560f-4e33-b057-9fde902dd485 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 21 21:07:20.575519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a14b9-560f-4e33-b057-9fde902dd485 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:20.584763 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 3/4] 10.222.0.13 () {58 vars in 1324 bytes} [Fri Oct 21 21:07:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 21:07:20.591183 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b26ba4-46a9-46f3-bcb0-49fb38d26719 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:20.606176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b26ba4-46a9-46f3-bcb0-49fb38d26719 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:20.610482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8aa06-5a97-44f9-a017-5c0c393de77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:20.611586 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8aa06-5a97-44f9-a017-5c0c393de77e admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:20.611872 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8aa06-5a97-44f9-a017-5c0c393de77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:20.612144 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8aa06-5a97-44f9-a017-5c0c393de77e admin admin] PATH_INFO: `/v3/projects/64a13af16ef54c01b2a28b4530062691` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:20.613553 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa8aa06-5a97-44f9-a017-5c0c393de77e admin admin] RBAC: Authorizing `identity:get_project(project_id=64a13af16ef54c01b2a28b4530062691)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:20.667788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2fa8aa06-5a97-44f9-a017-5c0c393de77e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 21 21:07:20.668126 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa8aa06-5a97-44f9-a017-5c0c393de77e admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:20.672621 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 2/5] 10.222.0.13 () {56 vars in 1207 bytes} [Fri Oct 21 21:07:20 2022] GET /identity/v3/projects/64a13af16ef54c01b2a28b4530062691 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:22.761484 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e3825-5728-4af9-b09e-1310c0b3439c None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:22.761908 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e3825-5728-4af9-b09e-1310c0b3439c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:22.762036 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e3825-5728-4af9-b09e-1310c0b3439c None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:22.766387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 4/6] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:22.771847 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b9434-f3ca-4cb5-83d7-b1d1f3e82acc None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:22.772209 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b9434-f3ca-4cb5-83d7-b1d1f3e82acc None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:22.772398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b9434-f3ca-4cb5-83d7-b1d1f3e82acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:22.792947 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-1e3b9434-f3ca-4cb5-83d7-b1d1f3e82acc None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:22.794453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-1e3b9434-f3ca-4cb5-83d7-b1d1f3e82acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:22.802083 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 3/7] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 21 21:07:23.070079 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e3825-5728-4af9-b09e-1310c0b3439c None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:23.082569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e3825-5728-4af9-b09e-1310c0b3439c None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:23.087327 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0462e7-ff15-4aed-af19-5db6c9376d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:23.087957 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0462e7-ff15-4aed-af19-5db6c9376d20 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:23.088241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0462e7-ff15-4aed-af19-5db6c9376d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:23.088514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0462e7-ff15-4aed-af19-5db6c9376d20 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:23.096153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0462e7-ff15-4aed-af19-5db6c9376d20 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:23.097931 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0462e7-ff15-4aed-af19-5db6c9376d20 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:23.107167 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 5/8] 10.222.0.13 () {58 vars in 1324 bytes} [Fri Oct 21 21:07:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 21:07:23.112543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b9434-f3ca-4cb5-83d7-b1d1f3e82acc None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:23.121959 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b9434-f3ca-4cb5-83d7-b1d1f3e82acc None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:23.126383 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b42f9-6209-49a4-9c75-7944911104f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:23.127003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b42f9-6209-49a4-9c75-7944911104f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:23.127283 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b42f9-6209-49a4-9c75-7944911104f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:23.127560 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b42f9-6209-49a4-9c75-7944911104f5 admin admin] PATH_INFO: `/v3/users/85a2a3bc91ec4aebbb1e136167a1d87f` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:23.128880 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b42f9-6209-49a4-9c75-7944911104f5 admin admin] RBAC: Authorizing `identity:get_user(user_id=85a2a3bc91ec4aebbb1e136167a1d87f)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:23.130699 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b42f9-6209-49a4-9c75-7944911104f5 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:23.132241 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 4/9] 10.222.0.13 () {56 vars in 1198 bytes} [Fri Oct 21 21:07:23 2022] GET /identity/v3/users/85a2a3bc91ec4aebbb1e136167a1d87f => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:25.307589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212a729-51aa-44a1-8af7-1828cebf1664 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:25.308216 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212a729-51aa-44a1-8af7-1828cebf1664 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:25.308216 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212a729-51aa-44a1-8af7-1828cebf1664 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:25.309462 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 6/10] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:25.314694 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439902aa-c1fc-4267-9420-8a50734a583b None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:25.315101 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439902aa-c1fc-4267-9420-8a50734a583b None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:25.315365 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439902aa-c1fc-4267-9420-8a50734a583b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:25.333675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-439902aa-c1fc-4267-9420-8a50734a583b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:25.334861 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-439902aa-c1fc-4267-9420-8a50734a583b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:25.342168 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c857aae-7fb5-4681-afe2-c22978601a80 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:25.342449 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c857aae-7fb5-4681-afe2-c22978601a80 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:25.342724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c857aae-7fb5-4681-afe2-c22978601a80 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:25.344115 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 7/11] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:07:25.349199 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bace08-4598-4492-af2f-6b3066cb89ad None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:25.349554 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bace08-4598-4492-af2f-6b3066cb89ad None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:25.349837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bace08-4598-4492-af2f-6b3066cb89ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:25.391805 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG passlib.handlers.bcrypt [None req-36bace08-4598-4492-af2f-6b3066cb89ad None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81305) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 21 21:07:25.392173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG passlib.handlers.bcrypt [None req-36bace08-4598-4492-af2f-6b3066cb89ad None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81305) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 21 21:07:25.413560 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 5/12] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:07:25.442148 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-36bace08-4598-4492-af2f-6b3066cb89ad None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:25.443755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-36bace08-4598-4492-af2f-6b3066cb89ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:25.541593 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 8/13] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:07:25.678650 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439902aa-c1fc-4267-9420-8a50734a583b None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:25.701537 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:25.780404 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439902aa-c1fc-4267-9420-8a50734a583b None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:25.808055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bace08-4598-4492-af2f-6b3066cb89ad None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:25.812537 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97979b-8212-4d11-909d-ba812ed4d502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:25.813162 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97979b-8212-4d11-909d-ba812ed4d502 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:25.813440 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97979b-8212-4d11-909d-ba812ed4d502 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:25.813704 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97979b-8212-4d11-909d-ba812ed4d502 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:25.824912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97979b-8212-4d11-909d-ba812ed4d502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:25.833788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-3c97979b-8212-4d11-909d-ba812ed4d502 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:07:25.833788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:25.833788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:25.833788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:25.833788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:25.833788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:25.833788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:25.833788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:25.833788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:25.833788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:25.833788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:25.833788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:25.833788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:25.833788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:25.833788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:25.833788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:25.834876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:25.834876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:25.834876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:25.834876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:25.834876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:25.834876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:25.834876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:25.834876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:25.834876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:25.834876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:25.834876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:25.834876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:25.834876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:25.834876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:25.834876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:25.834876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:25.836249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:25.836249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:25.836249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:25.836249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:25.836249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:25.836249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:25.836249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:25.836249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:25.836249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:25.836249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:25.836249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:07:25.836249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:07:25.836249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 6/14] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:07:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:07:25.840995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97979b-8212-4d11-909d-ba812ed4d502 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:25.908470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bace08-4598-4492-af2f-6b3066cb89ad None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:25.910746 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:25.920785 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:25.939867 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97979b-8212-4d11-909d-ba812ed4d502 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:25.946960 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db351a62-1e47-4844-975b-4f30f2a554ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:25.947858 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db351a62-1e47-4844-975b-4f30f2a554ee None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:25.948138 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db351a62-1e47-4844-975b-4f30f2a554ee None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:25.948405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db351a62-1e47-4844-975b-4f30f2a554ee None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:25.949363 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db351a62-1e47-4844-975b-4f30f2a554ee None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:25.950778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db351a62-1e47-4844-975b-4f30f2a554ee None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:25.961369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.sql.core [None req-db351a62-1e47-4844-975b-4f30f2a554ee None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 21 21:07:25.961369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 21 21:07:25.961369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [parameters: {'id': '5eadc984661c4119a394e19a890eaf95', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 21 21:07:25.961369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81305) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 21 21:07:25.972496 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003090a-9568-4ade-8589-3d180ce1e9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:25.973101 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003090a-9568-4ade-8589-3d180ce1e9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:25.973378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003090a-9568-4ade-8589-3d180ce1e9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:25.973647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003090a-9568-4ade-8589-3d180ce1e9e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:25.974304 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003090a-9568-4ade-8589-3d180ce1e9e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:25.974604 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003090a-9568-4ade-8589-3d180ce1e9e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:25.976015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003090a-9568-4ade-8589-3d180ce1e9e9 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:25.979979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-db351a62-1e47-4844-975b-4f30f2a554ee None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 21 21:07:25.979979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:25.979979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 21:07:25.979979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 21:07:25.979979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 21:07:25.979979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 21:07:25.979979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 21:07:25.979979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 21:07:25.979979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 21:07:25.979979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application conn.query(q) Oct 21 21:07:25.979979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 21:07:25.979979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 21:07:25.979979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 21:07:25.979979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application result.read() Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 21 21:07:25.980767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 21:07:25.981692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application next(self.gen) Oct 21 21:07:25.981692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 21 21:07:25.981692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application yield resource Oct 21 21:07:25.981692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 21:07:25.981692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application next(self.gen) Oct 21 21:07:25.981692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 21 21:07:25.981692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self.session.rollback() Oct 21 21:07:25.981692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 21 21:07:25.981692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self.force_reraise() Oct 21 21:07:25.981692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 21 21:07:25.981692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise self.value Oct 21 21:07:25.981692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 21 21:07:25.981692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 21 21:07:25.981692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 21 21:07:25.981692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application session.commit() Oct 21 21:07:25.981692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 21 21:07:25.981692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 21 21:07:25.982526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 21 21:07:25.982526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._prepare_impl() Oct 21 21:07:25.982526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 21 21:07:25.982526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self.session.flush() Oct 21 21:07:25.982526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 21 21:07:25.982526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._flush(objects) Oct 21 21:07:25.982526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 21 21:07:25.982526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 21 21:07:25.982526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 21 21:07:25.982526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application compat.raise_( Oct 21 21:07:25.982526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 21:07:25.982526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception Oct 21 21:07:25.982526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 21 21:07:25.982526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application flush_context.execute() Oct 21 21:07:25.982526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 21 21:07:25.982526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rec.execute(self) Oct 21 21:07:25.982526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 21 21:07:25.983428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 21 21:07:25.983428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 21 21:07:25.983428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 21 21:07:25.983428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 21 21:07:25.983428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 21 21:07:25.983428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 21 21:07:25.983428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 21 21:07:25.983428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 21 21:07:25.983428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 21 21:07:25.983428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 21 21:07:25.983428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 21 21:07:25.983428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 21 21:07:25.983428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 21 21:07:25.983428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 21 21:07:25.983428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 21 21:07:25.984243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 21:07:25.984243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception Oct 21 21:07:25.984243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 21:07:25.984243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 21:07:25.984243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 21:07:25.984243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 21:07:25.984243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 21:07:25.984243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 21:07:25.984243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 21:07:25.984243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application conn.query(q) Oct 21 21:07:25.984243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 21:07:25.984243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 21:07:25.984243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 21:07:25.984243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application result.read() Oct 21 21:07:25.984243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 21:07:25.984243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 21:07:25.984243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 21:07:25.985026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 21:07:25.985026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 21:07:25.985026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 21:07:25.985026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 21:07:25.985026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 21:07:25.985026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 21 21:07:25.985026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 21 21:07:25.985026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [parameters: {'id': '5eadc984661c4119a394e19a890eaf95', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 21 21:07:25.985026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 21 21:07:25.985026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:07:25.985026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:07:25.985026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:07:25.985026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:25.985026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 21 21:07:25.985026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 21 21:07:25.985730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 21 21:07:25.985730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 21 21:07:25.985730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 21 21:07:25.985730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:07:25.985730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:07:25.985730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:07:25.985730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:25.985730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:25.985730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:25.985730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:25.985730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:25.985730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:25.985730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:25.985730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:25.985730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:25.985730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:25.985730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:25.986609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 21 21:07:25.986609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 21 21:07:25.986609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:25.986609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:25.986609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 21 21:07:25.986609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 21 21:07:25.986609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:25.986609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:25.986609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 21 21:07:25.986609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 21 21:07:25.986609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 21 21:07:25.986609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:07:25.986609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 9/15] 10.222.0.13 () {60 vars in 1137 bytes} [Fri Oct 21 21:07:25 2022] POST /identity/v3/domains => generated 190 bytes in 174 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:07:25.987234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db351a62-1e47-4844-975b-4f30f2a554ee None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:25.988158 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 7/16] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:07:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:07:25.993530 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003090a-9568-4ade-8589-3d180ce1e9e9 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:26.088674 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db351a62-1e47-4844-975b-4f30f2a554ee None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:26.091904 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003090a-9568-4ade-8589-3d180ce1e9e9 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:26.122309 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec357d-d989-41ba-95bc-69a472eefa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:26.123156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec357d-d989-41ba-95bc-69a472eefa43 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:26.123437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec357d-d989-41ba-95bc-69a472eefa43 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:26.123741 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec357d-d989-41ba-95bc-69a472eefa43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:26.125223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e4307-b953-4527-8c14-8fc5b3345a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:26.125854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e4307-b953-4527-8c14-8fc5b3345a52 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:26.126136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e4307-b953-4527-8c14-8fc5b3345a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:26.126412 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e4307-b953-4527-8c14-8fc5b3345a52 None admin] PATH_INFO: `/v3/users/85a2a3bc91ec4aebbb1e136167a1d87f` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:26.127544 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e4307-b953-4527-8c14-8fc5b3345a52 None admin] RBAC: Authorizing `identity:get_user(user_id=85a2a3bc91ec4aebbb1e136167a1d87f)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:26.129070 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e4307-b953-4527-8c14-8fc5b3345a52 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:26.130637 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 8/17] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:25 2022] GET /identity/v3/users/85a2a3bc91ec4aebbb1e136167a1d87f => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:26.136108 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e4307-b953-4527-8c14-8fc5b3345a52 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:26.141835 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec357d-d989-41ba-95bc-69a472eefa43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:26.143304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec357d-d989-41ba-95bc-69a472eefa43 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:26.150592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:26.159096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-63ec357d-d989-41ba-95bc-69a472eefa43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:07:26.159096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:26.159096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:26.159096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:26.159096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:26.159096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:26.159096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:26.159096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:26.159096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:26.159096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:26.159096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:26.159096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:26.159096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:07:26.159096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:07:26.159096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:07:26.159096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:07:26.159993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:26.159993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:26.159993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:26.159993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:26.159993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:26.159993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:26.159993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:26.159993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:26.159993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:26.159993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:26.159993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:26.159993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:26.159993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:26.159993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:26.159993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:26.159993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:26.159993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:07:26.160783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:07:26.160783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:07:26.160783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:07:26.160783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 10/18] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:07:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:07:26.166408 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec357d-d989-41ba-95bc-69a472eefa43 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:26.191140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:26.234092 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e4307-b953-4527-8c14-8fc5b3345a52 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:26.268204 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec357d-d989-41ba-95bc-69a472eefa43 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:26.268992 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373b28e-49e2-4cd1-9d0d-d5100597cb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:26.269612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373b28e-49e2-4cd1-9d0d-d5100597cb99 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:26.269890 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373b28e-49e2-4cd1-9d0d-d5100597cb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:26.270160 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373b28e-49e2-4cd1-9d0d-d5100597cb99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:26.271269 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373b28e-49e2-4cd1-9d0d-d5100597cb99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:26.272744 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373b28e-49e2-4cd1-9d0d-d5100597cb99 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:26.274783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 9/19] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:07:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:07:26.280333 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373b28e-49e2-4cd1-9d0d-d5100597cb99 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:26.301746 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f39cd2-a1c2-4c3d-8925-2bc510dc78fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:26.302357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f39cd2-a1c2-4c3d-8925-2bc510dc78fe None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:26.302634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f39cd2-a1c2-4c3d-8925-2bc510dc78fe None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:26.302904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f39cd2-a1c2-4c3d-8925-2bc510dc78fe None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:26.303543 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f39cd2-a1c2-4c3d-8925-2bc510dc78fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:26.303870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f39cd2-a1c2-4c3d-8925-2bc510dc78fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:26.305253 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f39cd2-a1c2-4c3d-8925-2bc510dc78fe None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:26.322023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:07:26.360591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:26.380644 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373b28e-49e2-4cd1-9d0d-d5100597cb99 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:26.416366 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd8e8b-1d74-49e2-9969-7a9a978531fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:26.417038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd8e8b-1d74-49e2-9969-7a9a978531fd None admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:26.417315 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd8e8b-1d74-49e2-9969-7a9a978531fd None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:26.417617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd8e8b-1d74-49e2-9969-7a9a978531fd None admin] PATH_INFO: `/v3/domains/default/users/85a2a3bc91ec4aebbb1e136167a1d87f/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:26.419679 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd8e8b-1d74-49e2-9969-7a9a978531fd None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=85a2a3bc91ec4aebbb1e136167a1d87f, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:26.421140 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd8e8b-1d74-49e2-9969-7a9a978531fd None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:26.535747 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 10/21] 10.222.0.13 () {58 vars in 1361 bytes} [Fri Oct 21 21:07:26 2022] PUT /identity/v3/domains/default/users/85a2a3bc91ec4aebbb1e136167a1d87f/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:07:28.665006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51b40e-b7f7-4120-b4cf-6621ef13c641 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:28.665507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51b40e-b7f7-4120-b4cf-6621ef13c641 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:28.665578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51b40e-b7f7-4120-b4cf-6621ef13c641 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:28.667038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 12/22] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:28.673056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eda67-339c-4412-92b6-25dde70aed08 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:28.673526 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eda67-339c-4412-92b6-25dde70aed08 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:28.673599 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eda67-339c-4412-92b6-25dde70aed08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:28.700697 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-cf1eda67-339c-4412-92b6-25dde70aed08 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:28.702225 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-cf1eda67-339c-4412-92b6-25dde70aed08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:28.791110 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 11/23] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:07:29.317811 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008663f-93cb-4efc-ba09-c43fb7f26b84 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:29.318180 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008663f-93cb-4efc-ba09-c43fb7f26b84 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:29.318446 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008663f-93cb-4efc-ba09-c43fb7f26b84 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:29.321100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 13/24] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:29.326616 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc40854-c4f6-42b6-8114-21f736dc6231 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:29.326911 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc40854-c4f6-42b6-8114-21f736dc6231 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:29.327192 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc40854-c4f6-42b6-8114-21f736dc6231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:29.328807 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008663f-93cb-4efc-ba09-c43fb7f26b84 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:29.356578 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-dcc40854-c4f6-42b6-8114-21f736dc6231 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:29.357776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-dcc40854-c4f6-42b6-8114-21f736dc6231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:29.424096 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:29.426586 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:29.431483 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 12/25] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:29.433635 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55172a21-e138-4729-9016-8bae0c23fd98 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:29.436666 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55172a21-e138-4729-9016-8bae0c23fd98 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:29.436892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55172a21-e138-4729-9016-8bae0c23fd98 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:29.440952 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 13/26] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:29.442001 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd58a45-565c-49d9-b42f-1dad25cb9f1e None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:29.442287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd58a45-565c-49d9-b42f-1dad25cb9f1e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:29.442617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd58a45-565c-49d9-b42f-1dad25cb9f1e None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:29.448182 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 14/27] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:29.453157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e114c4-0c8b-4e2c-9bac-7038e43e1b05 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:29.453157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e114c4-0c8b-4e2c-9bac-7038e43e1b05 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:29.453157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e114c4-0c8b-4e2c-9bac-7038e43e1b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:29.464543 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008663f-93cb-4efc-ba09-c43fb7f26b84 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:29.468207 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-79e114c4-0c8b-4e2c-9bac-7038e43e1b05 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:29.469387 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-79e114c4-0c8b-4e2c-9bac-7038e43e1b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:29.532875 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a67d25-7b2d-4196-b9b7-c2d72411b79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:29.533514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a67d25-7b2d-4196-b9b7-c2d72411b79a None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:29.533793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a67d25-7b2d-4196-b9b7-c2d72411b79a None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:29.534065 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a67d25-7b2d-4196-b9b7-c2d72411b79a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:29.557082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a67d25-7b2d-4196-b9b7-c2d72411b79a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:29.558556 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a67d25-7b2d-4196-b9b7-c2d72411b79a None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:29.580633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-71a67d25-7b2d-4196-b9b7-c2d72411b79a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:07:29.580633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:29.580633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:29.580633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:29.580633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:29.580633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:29.580633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:29.580633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:29.580633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:29.580633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:29.580633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:29.580633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:29.580633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:07:29.580633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:07:29.580633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:07:29.580633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:07:29.581490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:29.581490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:29.581490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:29.581490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:29.581490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:29.581490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:29.581490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:29.581490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:29.581490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:29.581490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:29.581490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:29.581490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:29.581490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:29.581490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:29.581490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:29.581490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:29.581490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:07:29.582265 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:07:29.582265 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:07:29.582265 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:07:29.582265 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 14/28] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:07:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:07:29.582978 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b2f48-f252-4d51-8066-e8f6a0f164f8 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:29.583143 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b2f48-f252-4d51-8066-e8f6a0f164f8 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:29.583434 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b2f48-f252-4d51-8066-e8f6a0f164f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:29.604484 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-812b2f48-f252-4d51-8066-e8f6a0f164f8 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:29.605680 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-812b2f48-f252-4d51-8066-e8f6a0f164f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:29.607124 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 15/29] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:29.608205 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e114c4-0c8b-4e2c-9bac-7038e43e1b05 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:29.682231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 15/30] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:29.769645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b2f48-f252-4d51-8066-e8f6a0f164f8 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:29.819254 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e114c4-0c8b-4e2c-9bac-7038e43e1b05 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:29.843075 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:29.885245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:29.892154 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b2f48-f252-4d51-8066-e8f6a0f164f8 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:29.906929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eda4f8-15d1-483d-a809-2ff3cd2e6d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:29.907601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eda4f8-15d1-483d-a809-2ff3cd2e6d96 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:29.907866 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eda4f8-15d1-483d-a809-2ff3cd2e6d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:29.908139 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eda4f8-15d1-483d-a809-2ff3cd2e6d96 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:29.908779 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eda4f8-15d1-483d-a809-2ff3cd2e6d96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:29.909081 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eda4f8-15d1-483d-a809-2ff3cd2e6d96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:29.915947 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eda4f8-15d1-483d-a809-2ff3cd2e6d96 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:29.931962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628516ac-fcdd-45aa-9af6-1274409adb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:29.932918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628516ac-fcdd-45aa-9af6-1274409adb92 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:29.933404 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628516ac-fcdd-45aa-9af6-1274409adb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:29.933884 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628516ac-fcdd-45aa-9af6-1274409adb92 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:29.934744 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628516ac-fcdd-45aa-9af6-1274409adb92 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:29.936908 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628516ac-fcdd-45aa-9af6-1274409adb92 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:29.942935 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 16/31] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:29.951403 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eda4f8-15d1-483d-a809-2ff3cd2e6d96 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:30.084376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 16/32] 10.222.0.13 () {60 vars in 1131 bytes} [Fri Oct 21 21:07:29 2022] POST /identity/v3/roles => generated 225 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:07:30.118935 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628516ac-fcdd-45aa-9af6-1274409adb92 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:30.190018 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eda4f8-15d1-483d-a809-2ff3cd2e6d96 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:30.251086 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628516ac-fcdd-45aa-9af6-1274409adb92 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:30.279973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:30.281002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94357d7f-4c8a-484a-9e66-8e89c09a708f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:30.281912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94357d7f-4c8a-484a-9e66-8e89c09a708f None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:30.282017 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94357d7f-4c8a-484a-9e66-8e89c09a708f None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:30.282222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94357d7f-4c8a-484a-9e66-8e89c09a708f None admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:30.282887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94357d7f-4c8a-484a-9e66-8e89c09a708f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:30.284499 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94357d7f-4c8a-484a-9e66-8e89c09a708f None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:30.300864 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194ac0e-8c52-4e3f-a530-0ba5c3838036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:30.302048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194ac0e-8c52-4e3f-a530-0ba5c3838036 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:30.302506 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194ac0e-8c52-4e3f-a530-0ba5c3838036 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:30.302953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194ac0e-8c52-4e3f-a530-0ba5c3838036 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:30.323534 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194ac0e-8c52-4e3f-a530-0ba5c3838036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:30.326247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 17/33] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Oct 21 21:07:29 2022] POST /identity/v3/projects => generated 301 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:07:30.327356 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94357d7f-4c8a-484a-9e66-8e89c09a708f None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:30.330937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-e194ac0e-8c52-4e3f-a530-0ba5c3838036 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:07:30.330937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:30.330937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:30.330937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:30.330937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:30.330937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:30.330937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:30.330937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:30.330937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:30.330937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:30.330937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:30.330937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:30.330937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:30.330937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:30.330937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:30.330937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:30.331916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:30.331916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:30.331916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:30.331916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:30.331916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:30.331916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:30.331916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:30.331916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:30.331916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:30.331916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:30.331916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:30.331916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:30.331916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:30.331916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:30.331916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:30.331916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:30.332660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:30.332660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:30.332660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:30.332660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:30.332660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:30.332660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:30.332660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:30.332660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:30.332660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:30.332660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:30.332660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:07:30.332660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:07:30.335929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 17/34] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:07:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:07:30.339088 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194ac0e-8c52-4e3f-a530-0ba5c3838036 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:30.378141 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:30.457326 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94357d7f-4c8a-484a-9e66-8e89c09a708f None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:30.481761 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194ac0e-8c52-4e3f-a530-0ba5c3838036 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:30.502835 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b96610-9b41-476f-b609-6ba3566152bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:30.503830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b96610-9b41-476f-b609-6ba3566152bf None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:30.504286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b96610-9b41-476f-b609-6ba3566152bf None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:30.504762 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b96610-9b41-476f-b609-6ba3566152bf None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:30.505651 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b96610-9b41-476f-b609-6ba3566152bf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:30.507827 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b96610-9b41-476f-b609-6ba3566152bf None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:30.525158 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64061e5b-9f0e-4a22-bd76-3c7588b8b08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:30.525841 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64061e5b-9f0e-4a22-bd76-3c7588b8b08a None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:30.526119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64061e5b-9f0e-4a22-bd76-3c7588b8b08a None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:30.526398 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64061e5b-9f0e-4a22-bd76-3c7588b8b08a None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:30.527235 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64061e5b-9f0e-4a22-bd76-3c7588b8b08a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:30.527544 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64061e5b-9f0e-4a22-bd76-3c7588b8b08a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:30.529329 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64061e5b-9f0e-4a22-bd76-3c7588b8b08a None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:30.531658 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 18/35] 10.222.0.13 () {60 vars in 1131 bytes} [Fri Oct 21 21:07:30 2022] POST /identity/v3/roles => generated 219 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:07:30.548519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 18/36] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:07:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:07:30.554637 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c47bb-afab-4193-8d10-faeedd8c67c5 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:30.554959 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c47bb-afab-4193-8d10-faeedd8c67c5 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:30.555051 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64061e5b-9f0e-4a22-bd76-3c7588b8b08a None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:30.555199 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c47bb-afab-4193-8d10-faeedd8c67c5 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:30.557053 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 19/37] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:30.565106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e699f9-f202-43ac-ae35-bfcd08ad85f2 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:30.565419 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e699f9-f202-43ac-ae35-bfcd08ad85f2 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:30.565668 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e699f9-f202-43ac-ae35-bfcd08ad85f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:30.612276 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:30.612782 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-42e699f9-f202-43ac-ae35-bfcd08ad85f2 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:30.614044 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-42e699f9-f202-43ac-ae35-bfcd08ad85f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:30.691825 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64061e5b-9f0e-4a22-bd76-3c7588b8b08a None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:30.729658 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 20/38] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:30.755552 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fef7d-8cb6-4191-b7d5-e9b704d3f633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:30.756615 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fef7d-8cb6-4191-b7d5-e9b704d3f633 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:30.757094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fef7d-8cb6-4191-b7d5-e9b704d3f633 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:30.757641 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fef7d-8cb6-4191-b7d5-e9b704d3f633 None admin] PATH_INFO: `/v3/users/85a2a3bc91ec4aebbb1e136167a1d87f` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:30.759444 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fef7d-8cb6-4191-b7d5-e9b704d3f633 None admin] RBAC: Authorizing `identity:get_user(user_id=85a2a3bc91ec4aebbb1e136167a1d87f)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:30.761126 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fef7d-8cb6-4191-b7d5-e9b704d3f633 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:30.762964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 19/39] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:30 2022] GET /identity/v3/users/85a2a3bc91ec4aebbb1e136167a1d87f => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:07:30.769000 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e699f9-f202-43ac-ae35-bfcd08ad85f2 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:30.806431 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:30.884673 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e699f9-f202-43ac-ae35-bfcd08ad85f2 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:30.920456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62939df7-71d8-4ed4-9be6-6c1465dead15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:30.921484 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62939df7-71d8-4ed4-9be6-6c1465dead15 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:30.921963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62939df7-71d8-4ed4-9be6-6c1465dead15 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:30.922453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62939df7-71d8-4ed4-9be6-6c1465dead15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:30.923781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62939df7-71d8-4ed4-9be6-6c1465dead15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:30.925938 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62939df7-71d8-4ed4-9be6-6c1465dead15 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:30.926294 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780b629-3129-468a-b9ad-7177abfc6b59 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:30.926579 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780b629-3129-468a-b9ad-7177abfc6b59 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:30.926896 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780b629-3129-468a-b9ad-7177abfc6b59 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:30.928417 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 20/41] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:30.928641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 21/41] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:07:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:30.933499 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb136f86-d201-4cac-a535-1226abd2532e None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:30.934846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb136f86-d201-4cac-a535-1226abd2532e None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:30.934846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb136f86-d201-4cac-a535-1226abd2532e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:30.935043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62939df7-71d8-4ed4-9be6-6c1465dead15 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:30.953655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-eb136f86-d201-4cac-a535-1226abd2532e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:30.955187 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-eb136f86-d201-4cac-a535-1226abd2532e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:31.029152 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 21/42] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:31.030020 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb136f86-d201-4cac-a535-1226abd2532e None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:31.080267 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:31.143420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb136f86-d201-4cac-a535-1226abd2532e None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:31.147169 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:31.150647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62939df7-71d8-4ed4-9be6-6c1465dead15 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:31.178828 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8875d63-a48e-45ee-aa38-f05e1d3eff98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:31.179523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8875d63-a48e-45ee-aa38-f05e1d3eff98 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:31.179810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8875d63-a48e-45ee-aa38-f05e1d3eff98 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:31.180085 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8875d63-a48e-45ee-aa38-f05e1d3eff98 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:31.180750 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8875d63-a48e-45ee-aa38-f05e1d3eff98 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:31.182107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8875d63-a48e-45ee-aa38-f05e1d3eff98 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:31.185155 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af1e46-68ad-41ba-b40f-ca241e70c49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:31.185768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af1e46-68ad-41ba-b40f-ca241e70c49f None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:31.186054 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af1e46-68ad-41ba-b40f-ca241e70c49f None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:31.186337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af1e46-68ad-41ba-b40f-ca241e70c49f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:31.187829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af1e46-68ad-41ba-b40f-ca241e70c49f None admin] RBAC: Adding query filter params (role.id=c514b065b7604094b899c586f79b0c5b, scope.domain.id=default, user.id=85a2a3bc91ec4aebbb1e136167a1d87f) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:31.188164 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af1e46-68ad-41ba-b40f-ca241e70c49f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:31.189619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af1e46-68ad-41ba-b40f-ca241e70c49f None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:31.197951 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 22/43] 10.222.0.13 () {60 vars in 1131 bytes} [Fri Oct 21 21:07:31 2022] POST /identity/v3/roles => generated 223 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:07:31.204113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 22/44] 10.222.0.13 () {56 vars in 1322 bytes} [Fri Oct 21 21:07:30 2022] GET /identity/v3/role_assignments?user.id=85a2a3bc91ec4aebbb1e136167a1d87f&scope.domain.id=default&role.id=c514b065b7604094b899c586f79b0c5b => generated 530 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:31.299974 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8875d63-a48e-45ee-aa38-f05e1d3eff98 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:31.410666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8875d63-a48e-45ee-aa38-f05e1d3eff98 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:31.412237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:31.448321 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db342d-052f-4dcc-97cb-db1145c2bfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:31.449253 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db342d-052f-4dcc-97cb-db1145c2bfc7 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:31.449534 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db342d-052f-4dcc-97cb-db1145c2bfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:31.449807 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db342d-052f-4dcc-97cb-db1145c2bfc7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:31.450919 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db342d-052f-4dcc-97cb-db1145c2bfc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:31.452522 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db342d-052f-4dcc-97cb-db1145c2bfc7 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:31.454175 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 23/45] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:07:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:31.459927 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af1e46-68ad-41ba-b40f-ca241e70c49f None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:31.556866 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af1e46-68ad-41ba-b40f-ca241e70c49f None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:31.588941 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd728b4-1408-4d28-843f-f242912c5ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:31.589574 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd728b4-1408-4d28-843f-f242912c5ec6 None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:31.589858 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd728b4-1408-4d28-843f-f242912c5ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:31.590136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd728b4-1408-4d28-843f-f242912c5ec6 None admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:31.590794 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd728b4-1408-4d28-843f-f242912c5ec6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:31.592263 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd728b4-1408-4d28-843f-f242912c5ec6 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:31.617867 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 23/46] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Oct 21 21:07:31 2022] POST /identity/v3/projects => generated 312 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:07:33.738499 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724541e5-597d-4287-b0ff-c82f8d4e91f3 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:33.738876 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724541e5-597d-4287-b0ff-c82f8d4e91f3 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:33.739064 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724541e5-597d-4287-b0ff-c82f8d4e91f3 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:33.740430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 24/47] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:33.746235 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d04fb-8e0a-456f-9184-31547beeff20 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:33.746630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d04fb-8e0a-456f-9184-31547beeff20 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:33.746791 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d04fb-8e0a-456f-9184-31547beeff20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:33.764319 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-cf8d04fb-8e0a-456f-9184-31547beeff20 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:33.765519 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-cf8d04fb-8e0a-456f-9184-31547beeff20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:33.776819 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:33.831452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 24/48] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:34.151191 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724541e5-597d-4287-b0ff-c82f8d4e91f3 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:34.247695 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724541e5-597d-4287-b0ff-c82f8d4e91f3 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:34.280528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f018e-0d79-441d-87d9-bd12bb1a2b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:34.281148 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f018e-0d79-441d-87d9-bd12bb1a2b02 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:34.281430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f018e-0d79-441d-87d9-bd12bb1a2b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:34.281702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f018e-0d79-441d-87d9-bd12bb1a2b02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:34.282834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f018e-0d79-441d-87d9-bd12bb1a2b02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:34.284315 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f018e-0d79-441d-87d9-bd12bb1a2b02 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:34.285874 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 25/49] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:07:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:34.291742 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d04fb-8e0a-456f-9184-31547beeff20 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:34.385985 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d04fb-8e0a-456f-9184-31547beeff20 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:34.418725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31d96e-7dd5-4b84-aaee-3b8d60a4251e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:34.419287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31d96e-7dd5-4b84-aaee-3b8d60a4251e None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:34.419594 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31d96e-7dd5-4b84-aaee-3b8d60a4251e None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:34.419864 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31d96e-7dd5-4b84-aaee-3b8d60a4251e None admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:34.420523 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31d96e-7dd5-4b84-aaee-3b8d60a4251e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:34.422020 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31d96e-7dd5-4b84-aaee-3b8d60a4251e None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:34.441228 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:34.448573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 25/50] 10.222.0.13 () {60 vars in 1140 bytes} [Fri Oct 21 21:07:34 2022] POST /identity/v3/projects => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:07:36.590061 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee4a43-d6ba-45e1-9d72-03aa8e19fe29 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:36.590602 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee4a43-d6ba-45e1-9d72-03aa8e19fe29 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:36.590602 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee4a43-d6ba-45e1-9d72-03aa8e19fe29 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:36.591974 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 26/51] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:36.597161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c035f0-d982-47e9-93f0-44d7f67629ab None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:36.597564 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c035f0-d982-47e9-93f0-44d7f67629ab None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:36.597721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c035f0-d982-47e9-93f0-44d7f67629ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:36.615364 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-94c035f0-d982-47e9-93f0-44d7f67629ab None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:36.616598 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-94c035f0-d982-47e9-93f0-44d7f67629ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:36.679925 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 26/52] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:36.948081 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee4a43-d6ba-45e1-9d72-03aa8e19fe29 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:36.988320 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:37.047373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee4a43-d6ba-45e1-9d72-03aa8e19fe29 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:37.079243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6f24d-c14a-4068-bd65-e5feff019d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:37.079889 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6f24d-c14a-4068-bd65-e5feff019d26 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:37.080168 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6f24d-c14a-4068-bd65-e5feff019d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:37.080441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6f24d-c14a-4068-bd65-e5feff019d26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:37.081535 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6f24d-c14a-4068-bd65-e5feff019d26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:37.082969 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6f24d-c14a-4068-bd65-e5feff019d26 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:37.084544 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 27/53] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:07:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:37.090668 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c035f0-d982-47e9-93f0-44d7f67629ab None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:37.184724 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c035f0-d982-47e9-93f0-44d7f67629ab None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:37.209820 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:37.217527 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277e905-a66e-465a-ac68-4f44c5e0d1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:37.218126 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277e905-a66e-465a-ac68-4f44c5e0d1c3 None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:37.218405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277e905-a66e-465a-ac68-4f44c5e0d1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:37.218676 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277e905-a66e-465a-ac68-4f44c5e0d1c3 None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:37.219319 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277e905-a66e-465a-ac68-4f44c5e0d1c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:37.220799 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277e905-a66e-465a-ac68-4f44c5e0d1c3 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:37.247060 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 27/54] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:07:37 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:07:40.363756 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d9111-7f14-4b7d-a432-27cf6b1e08f0 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:40.364292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d9111-7f14-4b7d-a432-27cf6b1e08f0 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:40.364292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d9111-7f14-4b7d-a432-27cf6b1e08f0 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:40.365945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 28/55] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:40.376769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc877ef4-2d7f-4032-9372-82b81af75848 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:40.377193 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc877ef4-2d7f-4032-9372-82b81af75848 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:40.377373 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc877ef4-2d7f-4032-9372-82b81af75848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:40.400843 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-dc877ef4-2d7f-4032-9372-82b81af75848 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:40.402345 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-dc877ef4-2d7f-4032-9372-82b81af75848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:40.485816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 28/56] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:07:40.699604 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b8b36-5231-4e07-9d8b-6834e30dabcc None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:40.700283 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b8b36-5231-4e07-9d8b-6834e30dabcc None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:40.700761 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b8b36-5231-4e07-9d8b-6834e30dabcc None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:40.702288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 29/57] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:40.707518 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6f502-9515-4806-a1e0-eb7f34cbc809 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:40.708067 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6f502-9515-4806-a1e0-eb7f34cbc809 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:40.708531 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6f502-9515-4806-a1e0-eb7f34cbc809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:40.727863 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-d1f6f502-9515-4806-a1e0-eb7f34cbc809 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:40.729346 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d1f6f502-9515-4806-a1e0-eb7f34cbc809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:40.762861 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e077c82-b515-486b-974b-4a30963adeac None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:40.763139 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e077c82-b515-486b-974b-4a30963adeac None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:40.763410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e077c82-b515-486b-974b-4a30963adeac None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:40.764893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 30/58] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:07:40.773464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84f26a-dc22-4190-a9f3-5c4f6902200e None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:40.773740 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84f26a-dc22-4190-a9f3-5c4f6902200e None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:40.774170 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84f26a-dc22-4190-a9f3-5c4f6902200e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:40.805601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-3a84f26a-dc22-4190-a9f3-5c4f6902200e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:40.807017 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3a84f26a-dc22-4190-a9f3-5c4f6902200e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:40.812062 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 29/59] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:40.905448 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 31/60] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:40.910300 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6f502-9515-4806-a1e0-eb7f34cbc809 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:40.975134 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:41.053687 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6f502-9515-4806-a1e0-eb7f34cbc809 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:41.086038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee59dbd-1a1f-495b-bada-20f6dbe21ec3 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:41.086317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee59dbd-1a1f-495b-bada-20f6dbe21ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:41.086799 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee59dbd-1a1f-495b-bada-20f6dbe21ec3 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:41.088489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 32/61] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:41.097808 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21165ca6-450a-4512-b426-60800d2bf7fd None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:41.098379 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21165ca6-450a-4512-b426-60800d2bf7fd None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:41.098868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21165ca6-450a-4512-b426-60800d2bf7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:41.102117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac27872-5e7c-4408-bce3-8b177ff835d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:41.102760 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac27872-5e7c-4408-bce3-8b177ff835d6 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:41.103041 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac27872-5e7c-4408-bce3-8b177ff835d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:41.103322 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac27872-5e7c-4408-bce3-8b177ff835d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:41.125237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac27872-5e7c-4408-bce3-8b177ff835d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:41.130109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-cac27872-5e7c-4408-bce3-8b177ff835d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:07:41.130109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:41.130109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:41.130109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:41.130109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:41.130109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:41.130109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:41.130109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:41.130109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:41.130109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:41.130109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:41.130109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:41.130109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:41.130109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:41.130109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:41.130109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:41.130930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:41.130930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:41.130930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:41.130930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:41.130930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:41.130930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:41.130930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:41.130930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:41.130930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:41.130930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:41.130930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:41.130930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:41.130930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:41.130930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:41.130930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:41.130930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:41.132243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:41.132243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:41.132243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:41.132243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:41.132243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:41.132243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:41.132243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:41.132243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:41.132243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:41.132243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:41.132243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:07:41.132243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:07:41.135006 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 30/62] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:07:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:07:41.135991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-21165ca6-450a-4512-b426-60800d2bf7fd None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:41.137184 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-21165ca6-450a-4512-b426-60800d2bf7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:41.142023 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac27872-5e7c-4408-bce3-8b177ff835d6 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:41.178671 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:41.256872 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac27872-5e7c-4408-bce3-8b177ff835d6 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:41.259259 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:41.293500 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6356bf2-c0cc-4806-a6ba-b4caf6b58b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:41.294117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6356bf2-c0cc-4806-a6ba-b4caf6b58b9d None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:41.294401 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6356bf2-c0cc-4806-a6ba-b4caf6b58b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:41.294679 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6356bf2-c0cc-4806-a6ba-b4caf6b58b9d None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:41.295349 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6356bf2-c0cc-4806-a6ba-b4caf6b58b9d None admin] RBAC: Adding query filter params (name=member) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:41.295687 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6356bf2-c0cc-4806-a6ba-b4caf6b58b9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:41.297111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6356bf2-c0cc-4806-a6ba-b4caf6b58b9d None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:41.305429 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 31/63] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:07:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:41.306676 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c060e-ec74-46b2-ab8b-c710fa75759f None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:41.306962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c060e-ec74-46b2-ab8b-c710fa75759f None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:41.307239 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c060e-ec74-46b2-ab8b-c710fa75759f None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:41.308778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 32/64] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:41.309485 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c060e-ec74-46b2-ab8b-c710fa75759f None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:41.324777 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 33/65] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:41.328559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21165ca6-450a-4512-b426-60800d2bf7fd None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:41.419303 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c060e-ec74-46b2-ab8b-c710fa75759f None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:41.432663 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21165ca6-450a-4512-b426-60800d2bf7fd None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:41.456754 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d693c-39f7-4892-94c3-3952e1af5441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:41.457647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d693c-39f7-4892-94c3-3952e1af5441 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:41.458216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d693c-39f7-4892-94c3-3952e1af5441 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:41.458759 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d693c-39f7-4892-94c3-3952e1af5441 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:41.466663 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc110d-1274-40b5-a670-45bfd9abe864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:41.467297 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc110d-1274-40b5-a670-45bfd9abe864 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:41.467598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc110d-1274-40b5-a670-45bfd9abe864 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:41.467879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc110d-1274-40b5-a670-45bfd9abe864 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:41.471002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d693c-39f7-4892-94c3-3952e1af5441 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:41.477763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-839d693c-39f7-4892-94c3-3952e1af5441 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:07:41.477763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:41.477763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:41.477763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:41.477763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:41.477763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:41.477763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:41.477763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:41.477763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:41.477763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:41.477763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:41.477763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:41.477763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:41.477763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:41.477763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:41.477763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:41.478790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:41.478790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:41.478790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:41.478790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:41.478790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:41.478790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:41.478790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:41.478790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:41.478790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:41.478790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:41.478790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:41.478790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:41.478790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:41.478790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:41.478790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:41.478790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:41.479559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:41.479559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:41.479559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:41.479559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:41.479559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:41.479559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:41.479559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:41.479559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:41.479559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:41.479559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:41.479559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:07:41.479559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:07:41.480245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 33/66] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:07:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:07:41.480773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d693c-39f7-4892-94c3-3952e1af5441 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:41.480992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc110d-1274-40b5-a670-45bfd9abe864 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:41.486188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-03fc110d-1274-40b5-a670-45bfd9abe864 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:07:41.486188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:41.486188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:41.486188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:41.486188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:41.486188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:41.486188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:41.486188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:41.486188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:41.486188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:41.486188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:41.486188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:41.486188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:41.486188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:41.486188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:41.486188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:41.487010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:41.487010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:41.487010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:41.487010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:41.487010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:41.487010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:41.487010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:41.487010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:41.487010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:41.487010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:41.487010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:41.487010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:41.487010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:41.487010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:41.487010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:41.487010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:41.487787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:41.487787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:41.487787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:41.487787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:41.487787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:41.487787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:41.487787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:41.487787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:41.487787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:41.487787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:41.487787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:07:41.487787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:07:41.487787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 34/67] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Oct 21 21:07:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:07:41.493438 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c5f2e-9d0b-4bc9-a433-3a5ba446690b None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:41.493732 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c5f2e-9d0b-4bc9-a433-3a5ba446690b None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:41.494014 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c5f2e-9d0b-4bc9-a433-3a5ba446690b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:41.512578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-8e8c5f2e-9d0b-4bc9-a433-3a5ba446690b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:41.513376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:41.513720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8e8c5f2e-9d0b-4bc9-a433-3a5ba446690b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:41.536503 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:41.589358 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d693c-39f7-4892-94c3-3952e1af5441 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:41.591740 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 35/68] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:41.592695 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c5f2e-9d0b-4bc9-a433-3a5ba446690b None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:41.625180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40bbb6-1710-4df4-9fd1-647b187b73bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:41.625872 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40bbb6-1710-4df4-9fd1-647b187b73bd None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:41.626158 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40bbb6-1710-4df4-9fd1-647b187b73bd None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:41.626437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40bbb6-1710-4df4-9fd1-647b187b73bd None admin] PATH_INFO: `/v3/users/8e8963c7ee674a639acccd9a093fe9ee` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:41.644144 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c40bbb6-1710-4df4-9fd1-647b187b73bd None admin] RBAC: Authorizing `identity:get_user(user_id=8e8963c7ee674a639acccd9a093fe9ee)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:41.646079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c40bbb6-1710-4df4-9fd1-647b187b73bd None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:41.652567 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 34/69] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:41 2022] GET /identity/v3/users/8e8963c7ee674a639acccd9a093fe9ee => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:41.652567 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40bbb6-1710-4df4-9fd1-647b187b73bd None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:41.691883 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c5f2e-9d0b-4bc9-a433-3a5ba446690b None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:41.728402 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948de1d-33c0-4ab5-b924-471e8ee0fec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:41.729050 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948de1d-33c0-4ab5-b924-471e8ee0fec1 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:41.729542 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948de1d-33c0-4ab5-b924-471e8ee0fec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:41.729822 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948de1d-33c0-4ab5-b924-471e8ee0fec1 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:41.730492 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948de1d-33c0-4ab5-b924-471e8ee0fec1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:41.730805 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948de1d-33c0-4ab5-b924-471e8ee0fec1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:41.732252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948de1d-33c0-4ab5-b924-471e8ee0fec1 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:41.741041 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 36/70] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:07:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:41.746098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948de1d-33c0-4ab5-b924-471e8ee0fec1 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:41.752896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:41.766722 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:41.769971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40bbb6-1710-4df4-9fd1-647b187b73bd None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:41.805444 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6c77e-cbba-4c15-9159-2a773c03c314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:41.806054 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6c77e-cbba-4c15-9159-2a773c03c314 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:41.806337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6c77e-cbba-4c15-9159-2a773c03c314 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:41.806616 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6c77e-cbba-4c15-9159-2a773c03c314 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:41.807274 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6c77e-cbba-4c15-9159-2a773c03c314 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:41.807592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6c77e-cbba-4c15-9159-2a773c03c314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:41.808981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6c77e-cbba-4c15-9159-2a773c03c314 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:41.816835 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 35/71] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:07:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:41.817611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6c77e-cbba-4c15-9159-2a773c03c314 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:41.858092 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948de1d-33c0-4ab5-b924-471e8ee0fec1 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:41.892058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f01f48-5b35-4080-b0d9-f8dbfb28fbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:41.892694 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f01f48-5b35-4080-b0d9-f8dbfb28fbc4 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:41.892975 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f01f48-5b35-4080-b0d9-f8dbfb28fbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:41.893252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f01f48-5b35-4080-b0d9-f8dbfb28fbc4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:41.894367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f01f48-5b35-4080-b0d9-f8dbfb28fbc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:41.895832 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f01f48-5b35-4080-b0d9-f8dbfb28fbc4 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:41.897952 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 37/72] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:07:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:41.898745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f01f48-5b35-4080-b0d9-f8dbfb28fbc4 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:41.921182 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6c77e-cbba-4c15-9159-2a773c03c314 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:41.954891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfecc7b-0181-458d-b880-ec287d225a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:41.955524 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfecc7b-0181-458d-b880-ec287d225a22 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:41.955834 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfecc7b-0181-458d-b880-ec287d225a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:41.956108 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfecc7b-0181-458d-b880-ec287d225a22 None admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:41.957235 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfecc7b-0181-458d-b880-ec287d225a22 None admin] RBAC: Authorizing `identity:get_project(project_id=dd6450bb3add4920a802300916dc915c)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:41.958703 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfecc7b-0181-458d-b880-ec287d225a22 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:41.960926 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 36/73] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:07:41 2022] GET /identity/v3/projects/dd6450bb3add4920a802300916dc915c => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:41.961682 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfecc7b-0181-458d-b880-ec287d225a22 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:41.978302 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:41.983386 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:41.999445 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f01f48-5b35-4080-b0d9-f8dbfb28fbc4 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:42.033060 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79199612-ee12-47cc-9748-9b6405abd2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:42.033623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79199612-ee12-47cc-9748-9b6405abd2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:42.033906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79199612-ee12-47cc-9748-9b6405abd2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:42.034183 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79199612-ee12-47cc-9748-9b6405abd2f8 None admin] PATH_INFO: `/v3/users/85a2a3bc91ec4aebbb1e136167a1d87f` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:42.035311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79199612-ee12-47cc-9748-9b6405abd2f8 None admin] RBAC: Authorizing `identity:get_user(user_id=85a2a3bc91ec4aebbb1e136167a1d87f)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:42.036782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79199612-ee12-47cc-9748-9b6405abd2f8 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:42.038386 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 38/74] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:41 2022] GET /identity/v3/users/85a2a3bc91ec4aebbb1e136167a1d87f => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:42.039143 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79199612-ee12-47cc-9748-9b6405abd2f8 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:42.059832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfecc7b-0181-458d-b880-ec287d225a22 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:42.092189 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6ce24-cc74-4e0a-bddb-112bf69ee1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:42.092820 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6ce24-cc74-4e0a-bddb-112bf69ee1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:42.093105 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6ce24-cc74-4e0a-bddb-112bf69ee1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:42.093383 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6ce24-cc74-4e0a-bddb-112bf69ee1e2 None admin] PATH_INFO: `/v3/users/8e8963c7ee674a639acccd9a093fe9ee` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:42.094516 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6ce24-cc74-4e0a-bddb-112bf69ee1e2 None admin] RBAC: Authorizing `identity:get_user(user_id=8e8963c7ee674a639acccd9a093fe9ee)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:42.096007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6ce24-cc74-4e0a-bddb-112bf69ee1e2 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:42.097597 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 37/75] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:41 2022] GET /identity/v3/users/8e8963c7ee674a639acccd9a093fe9ee => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:42.098332 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6ce24-cc74-4e0a-bddb-112bf69ee1e2 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:42.144833 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79199612-ee12-47cc-9748-9b6405abd2f8 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:42.176673 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:42.176914 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112ff30-d83f-4836-a3da-d1b978439ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:42.177543 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112ff30-d83f-4836-a3da-d1b978439ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:42.177824 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112ff30-d83f-4836-a3da-d1b978439ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:42.178100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112ff30-d83f-4836-a3da-d1b978439ee4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:42.188850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112ff30-d83f-4836-a3da-d1b978439ee4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:42.194110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-f112ff30-d83f-4836-a3da-d1b978439ee4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:07:42.194110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:42.194110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:42.194110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:42.194110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:42.194110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:42.194110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:42.194110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:42.194110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:42.194110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:42.194110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:42.194110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:42.194110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:42.194110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:42.194110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:42.194110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:42.195460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:42.195460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:42.195460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:42.195460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:42.195460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:42.195460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:42.195460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:42.195460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:42.195460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:42.195460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:42.195460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:42.195460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:42.195460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:42.195460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:42.195460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:42.195460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:42.196273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:42.196273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:42.196273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:42.196273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:42.196273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:42.196273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:42.196273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:42.196273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:42.196273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:42.196273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:42.196273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:07:42.196273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:07:42.196273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 39/76] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:07:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:07:42.196886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112ff30-d83f-4836-a3da-d1b978439ee4 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:42.197426 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6ce24-cc74-4e0a-bddb-112bf69ee1e2 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:42.212065 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:42.230623 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3babe5-d798-4438-9a50-6d5940a6e459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:42.231236 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3babe5-d798-4438-9a50-6d5940a6e459 None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:42.231518 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3babe5-d798-4438-9a50-6d5940a6e459 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:42.231818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3babe5-d798-4438-9a50-6d5940a6e459 None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:42.232456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3babe5-d798-4438-9a50-6d5940a6e459 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:42.233917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3babe5-d798-4438-9a50-6d5940a6e459 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:42.254916 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 38/77] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:07:42 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:07:42.255748 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3babe5-d798-4438-9a50-6d5940a6e459 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:42.300469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112ff30-d83f-4836-a3da-d1b978439ee4 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:42.336657 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff4457-ccee-41ab-91ee-9c78cf2bc09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:42.337348 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff4457-ccee-41ab-91ee-9c78cf2bc09c None admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:42.337625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff4457-ccee-41ab-91ee-9c78cf2bc09c None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:42.337899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff4457-ccee-41ab-91ee-9c78cf2bc09c None admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c/users/8e8963c7ee674a639acccd9a093fe9ee/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:42.339963 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff4457-ccee-41ab-91ee-9c78cf2bc09c None admin] RBAC: Authorizing `identity:create_grant(project_id=dd6450bb3add4920a802300916dc915c, user_id=8e8963c7ee674a639acccd9a093fe9ee, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:42.341429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff4457-ccee-41ab-91ee-9c78cf2bc09c None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:42.358074 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 40/78] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:07:42 2022] PUT /identity/v3/projects/dd6450bb3add4920a802300916dc915c/users/8e8963c7ee674a639acccd9a093fe9ee/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:07:42.358674 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff4457-ccee-41ab-91ee-9c78cf2bc09c None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:42.384133 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3babe5-d798-4438-9a50-6d5940a6e459 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:42.429650 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4f21b-bd8c-4742-8d81-7ce52953db9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:42.430468 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4f21b-bd8c-4742-8d81-7ce52953db9b None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:42.430907 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4f21b-bd8c-4742-8d81-7ce52953db9b None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:42.431259 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4f21b-bd8c-4742-8d81-7ce52953db9b None admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:42.432680 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4f21b-bd8c-4742-8d81-7ce52953db9b None admin] RBAC: Authorizing `identity:get_project(project_id=dd6450bb3add4920a802300916dc915c)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:42.435316 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4f21b-bd8c-4742-8d81-7ce52953db9b None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:42.437290 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 39/79] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:07:42 2022] GET /identity/v3/projects/dd6450bb3add4920a802300916dc915c => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:42.438717 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4f21b-bd8c-4742-8d81-7ce52953db9b None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:42.449062 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:42.477353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:42.525929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff4457-ccee-41ab-91ee-9c78cf2bc09c None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:42.574718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb4571-6c38-47a1-8273-08bad841c838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:42.575413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb4571-6c38-47a1-8273-08bad841c838 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:42.575656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb4571-6c38-47a1-8273-08bad841c838 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:42.575934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb4571-6c38-47a1-8273-08bad841c838 None admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:42.577613 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb4571-6c38-47a1-8273-08bad841c838 None admin] RBAC: Authorizing `identity:get_project(project_id=dd6450bb3add4920a802300916dc915c)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:42.579080 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb4571-6c38-47a1-8273-08bad841c838 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:42.580758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 41/80] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:07:42 2022] GET /identity/v3/projects/dd6450bb3add4920a802300916dc915c => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:42.581503 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb4571-6c38-47a1-8273-08bad841c838 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:42.582582 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4f21b-bd8c-4742-8d81-7ce52953db9b None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:42.629059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202e396-d4e9-40fa-8bff-a85c4b5ebb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:42.629680 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202e396-d4e9-40fa-8bff-a85c4b5ebb6a None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:42.629959 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202e396-d4e9-40fa-8bff-a85c4b5ebb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:42.630230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202e396-d4e9-40fa-8bff-a85c4b5ebb6a None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:42.630894 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202e396-d4e9-40fa-8bff-a85c4b5ebb6a None admin] RBAC: Adding query filter params (name=member) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:42.631195 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202e396-d4e9-40fa-8bff-a85c4b5ebb6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:42.632721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202e396-d4e9-40fa-8bff-a85c4b5ebb6a None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:42.642293 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 40/81] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:07:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:42.642904 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202e396-d4e9-40fa-8bff-a85c4b5ebb6a None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:42.719402 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb4571-6c38-47a1-8273-08bad841c838 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:42.741991 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:42.757163 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2f35b-1c3b-46ac-a072-6ebda56d0ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:42.758120 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2f35b-1c3b-46ac-a072-6ebda56d0ecb None admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:42.758406 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2f35b-1c3b-46ac-a072-6ebda56d0ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:42.758683 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2f35b-1c3b-46ac-a072-6ebda56d0ecb None admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c/users/85a2a3bc91ec4aebbb1e136167a1d87f/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:42.762252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2f35b-1c3b-46ac-a072-6ebda56d0ecb None admin] RBAC: Authorizing `identity:create_grant(project_id=dd6450bb3add4920a802300916dc915c, user_id=85a2a3bc91ec4aebbb1e136167a1d87f, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:42.764066 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2f35b-1c3b-46ac-a072-6ebda56d0ecb None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:42.781417 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 42/82] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:07:42 2022] PUT /identity/v3/projects/dd6450bb3add4920a802300916dc915c/users/85a2a3bc91ec4aebbb1e136167a1d87f/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:07:42.782461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2f35b-1c3b-46ac-a072-6ebda56d0ecb None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:42.785292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:42.790510 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202e396-d4e9-40fa-8bff-a85c4b5ebb6a None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:42.846935 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47da9e-d7c3-419c-bdde-f06df3c278aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:42.847730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47da9e-d7c3-419c-bdde-f06df3c278aa None admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:42.848009 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47da9e-d7c3-419c-bdde-f06df3c278aa None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:42.848283 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47da9e-d7c3-419c-bdde-f06df3c278aa None admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c/users/8e8963c7ee674a639acccd9a093fe9ee/roles/01dbc90c2ce041afa161e621bd524341` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:42.853895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47da9e-d7c3-419c-bdde-f06df3c278aa None admin] RBAC: Authorizing `identity:create_grant(project_id=dd6450bb3add4920a802300916dc915c, user_id=8e8963c7ee674a639acccd9a093fe9ee, role_id=01dbc90c2ce041afa161e621bd524341)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:42.855709 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47da9e-d7c3-419c-bdde-f06df3c278aa None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:42.878898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 41/83] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:07:42 2022] PUT /identity/v3/projects/dd6450bb3add4920a802300916dc915c/users/8e8963c7ee674a639acccd9a093fe9ee/roles/01dbc90c2ce041afa161e621bd524341 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:07:42.908833 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2f35b-1c3b-46ac-a072-6ebda56d0ecb None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:42.950675 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a3e86-9233-429f-a956-4748af04a360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:42.951331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a3e86-9233-429f-a956-4748af04a360 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:42.951633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a3e86-9233-429f-a956-4748af04a360 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:42.951908 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a3e86-9233-429f-a956-4748af04a360 None admin] PATH_INFO: `/v3/users/8e8963c7ee674a639acccd9a093fe9ee` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:42.953116 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a3e86-9233-429f-a956-4748af04a360 None admin] RBAC: Authorizing `identity:get_user(user_id=8e8963c7ee674a639acccd9a093fe9ee)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:42.954583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a3e86-9233-429f-a956-4748af04a360 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:42.956565 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 43/84] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:42 2022] GET /identity/v3/users/8e8963c7ee674a639acccd9a093fe9ee => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:42.962761 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47da9e-d7c3-419c-bdde-f06df3c278aa None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:43.114563 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47da9e-d7c3-419c-bdde-f06df3c278aa None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:43.146100 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:43.157125 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eafadf-cd61-4abb-98d5-166687d653d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:43.157841 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eafadf-cd61-4abb-98d5-166687d653d1 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:43.158129 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eafadf-cd61-4abb-98d5-166687d653d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:43.158406 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eafadf-cd61-4abb-98d5-166687d653d1 None admin] PATH_INFO: `/v3/projects/9dfec435f3e7456aa2d5b896994cdec4` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:43.159701 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eafadf-cd61-4abb-98d5-166687d653d1 None admin] RBAC: Authorizing `identity:get_project(project_id=9dfec435f3e7456aa2d5b896994cdec4)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:43.161315 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eafadf-cd61-4abb-98d5-166687d653d1 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:43.163296 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 42/85] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:07:42 2022] GET /identity/v3/projects/9dfec435f3e7456aa2d5b896994cdec4 => generated 312 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:43.168895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a3e86-9233-429f-a956-4748af04a360 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:43.252287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:43.288795 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a3e86-9233-429f-a956-4748af04a360 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:43.330519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982a9a9-9a08-4723-a6b4-59bcbb97a7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:43.331194 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982a9a9-9a08-4723-a6b4-59bcbb97a7a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:43.331478 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982a9a9-9a08-4723-a6b4-59bcbb97a7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:43.331909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982a9a9-9a08-4723-a6b4-59bcbb97a7a1 None admin] PATH_INFO: `/v3/projects/9dfec435f3e7456aa2d5b896994cdec4/users/8e8963c7ee674a639acccd9a093fe9ee/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:43.333935 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982a9a9-9a08-4723-a6b4-59bcbb97a7a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9dfec435f3e7456aa2d5b896994cdec4, user_id=8e8963c7ee674a639acccd9a093fe9ee, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:43.335971 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982a9a9-9a08-4723-a6b4-59bcbb97a7a1 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:43.357503 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 44/86] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:07:43 2022] PUT /identity/v3/projects/9dfec435f3e7456aa2d5b896994cdec4/users/8e8963c7ee674a639acccd9a093fe9ee/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:07:45.255040 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d4027-23f5-49f2-a5d2-7fc13c0fb100 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:45.255547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d4027-23f5-49f2-a5d2-7fc13c0fb100 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:45.255645 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d4027-23f5-49f2-a5d2-7fc13c0fb100 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:45.257548 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 43/87] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:45.262938 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34fd7e-cec9-4f26-82f6-0142d33f9912 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:45.263338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34fd7e-cec9-4f26-82f6-0142d33f9912 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:45.263498 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34fd7e-cec9-4f26-82f6-0142d33f9912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:45.289187 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-5f34fd7e-cec9-4f26-82f6-0142d33f9912 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:45.290387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-5f34fd7e-cec9-4f26-82f6-0142d33f9912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:45.365776 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 45/88] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:45.589998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef96354-b1ae-43f0-922f-38a695c3fb52 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:45.590200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef96354-b1ae-43f0-922f-38a695c3fb52 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:45.590483 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef96354-b1ae-43f0-922f-38a695c3fb52 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:45.591889 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 44/89] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:45.597206 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58631668-8a5d-4997-8ec7-fd365c3c5fb4 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:45.597494 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58631668-8a5d-4997-8ec7-fd365c3c5fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:45.597768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58631668-8a5d-4997-8ec7-fd365c3c5fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:45.620566 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-58631668-8a5d-4997-8ec7-fd365c3c5fb4 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:45.621778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-58631668-8a5d-4997-8ec7-fd365c3c5fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:45.698925 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:45.711203 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 46/90] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:45.807231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5679ab-c101-4240-a579-e0bf8edbce06 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:45.807542 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5679ab-c101-4240-a579-e0bf8edbce06 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:45.807817 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5679ab-c101-4240-a579-e0bf8edbce06 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:45.809597 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 45/91] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:45.814717 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa0e88-e55e-4686-b7d0-2ea1eff9cd0f None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:45.815007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa0e88-e55e-4686-b7d0-2ea1eff9cd0f None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:45.815282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa0e88-e55e-4686-b7d0-2ea1eff9cd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:45.833522 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-80aa0e88-e55e-4686-b7d0-2ea1eff9cd0f None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:45.835224 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-80aa0e88-e55e-4686-b7d0-2ea1eff9cd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:45.904543 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 47/92] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:45.919470 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5679ab-c101-4240-a579-e0bf8edbce06 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:46.053822 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5679ab-c101-4240-a579-e0bf8edbce06 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:46.071922 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a11fd-e718-4183-9cce-75915a7c30b2 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:46.072523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a11fd-e718-4183-9cce-75915a7c30b2 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:46.073015 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a11fd-e718-4183-9cce-75915a7c30b2 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:46.074542 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 48/93] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:46.079700 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b93b8-ff85-471f-b4f4-394c8af891ef None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:46.080218 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b93b8-ff85-471f-b4f4-394c8af891ef None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:46.080686 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b93b8-ff85-471f-b4f4-394c8af891ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:46.104808 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ec956-82cf-4edf-b21d-d8bd0236d9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:46.105792 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ec956-82cf-4edf-b21d-d8bd0236d9d6 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:46.106264 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ec956-82cf-4edf-b21d-d8bd0236d9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:46.106722 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ec956-82cf-4edf-b21d-d8bd0236d9d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:46.107822 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-df3b93b8-ff85-471f-b4f4-394c8af891ef None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:46.110455 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-df3b93b8-ff85-471f-b4f4-394c8af891ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:46.126647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ec956-82cf-4edf-b21d-d8bd0236d9d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:46.134813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-a47ec956-82cf-4edf-b21d-d8bd0236d9d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 21:07:46.134813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:46.134813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:46.134813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:46.134813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:46.134813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:46.134813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:46.134813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:46.134813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:46.134813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:46.134813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:46.134813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:46.134813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:46.134813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:46.134813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:46.134813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:46.135652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:46.135652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:46.135652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:46.135652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:46.135652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:46.135652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:46.135652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:46.135652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:46.135652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:46.135652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:46.135652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:46.135652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:46.135652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:46.135652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:46.135652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:46.135652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:46.136415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:46.136415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:46.136415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:46.136415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:46.136415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:46.136415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:46.136415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:46.136415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:46.136415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:46.136415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:46.136415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 21:07:46.136415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:07:46.139282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 46/94] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:07:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:07:46.140693 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ec956-82cf-4edf-b21d-d8bd0236d9d6 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:46.203937 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:46.273966 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ec956-82cf-4edf-b21d-d8bd0236d9d6 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:46.294249 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 49/95] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:07:46.295967 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3b93b8-ff85-471f-b4f4-394c8af891ef None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:46.341492 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79ad4f-bd1d-4b0d-8581-546ad216c74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:46.343106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79ad4f-bd1d-4b0d-8581-546ad216c74b None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:46.343800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79ad4f-bd1d-4b0d-8581-546ad216c74b None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:46.344295 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79ad4f-bd1d-4b0d-8581-546ad216c74b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:46.345665 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79ad4f-bd1d-4b0d-8581-546ad216c74b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:46.347386 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79ad4f-bd1d-4b0d-8581-546ad216c74b None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:46.349930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 47/96] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:07:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:46.351841 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79ad4f-bd1d-4b0d-8581-546ad216c74b None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:46.379819 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:46.434839 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3b93b8-ff85-471f-b4f4-394c8af891ef None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:46.471981 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a73a9-c39a-4b1b-a7fe-3f6ac54e19a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:46.473017 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a73a9-c39a-4b1b-a7fe-3f6ac54e19a3 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:46.473555 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a73a9-c39a-4b1b-a7fe-3f6ac54e19a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:46.474018 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a73a9-c39a-4b1b-a7fe-3f6ac54e19a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:46.474865 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a73a9-c39a-4b1b-a7fe-3f6ac54e19a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:46.475902 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a73a9-c39a-4b1b-a7fe-3f6ac54e19a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:46.476713 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79ad4f-bd1d-4b0d-8581-546ad216c74b None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:46.477670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a73a9-c39a-4b1b-a7fe-3f6ac54e19a3 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:46.486767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 50/97] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:07:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:46.488664 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7e9b1-c30e-4adb-9f79-9096e285f688 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:46.488952 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7e9b1-c30e-4adb-9f79-9096e285f688 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:46.489228 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7e9b1-c30e-4adb-9f79-9096e285f688 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:46.490730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 51/98] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:46.491736 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7e9b1-c30e-4adb-9f79-9096e285f688 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:46.504826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:46.519181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87569c5c-d825-4189-844b-3758805046e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:46.520333 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87569c5c-d825-4189-844b-3758805046e0 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:46.521015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87569c5c-d825-4189-844b-3758805046e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:46.521439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87569c5c-d825-4189-844b-3758805046e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:46.533804 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87569c5c-d825-4189-844b-3758805046e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:46.539323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-87569c5c-d825-4189-844b-3758805046e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:07:46.539323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:46.539323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:46.539323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:46.539323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:46.539323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:46.539323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:46.539323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:46.539323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:46.539323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:46.539323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:46.539323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:46.539323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:46.539323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:46.539323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:46.539323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:46.540224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:46.540224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:46.540224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:46.540224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:46.540224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:46.540224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:46.540224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:46.540224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:46.540224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:46.540224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:46.540224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:46.540224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:46.540224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:46.540224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:46.540224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:46.540224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:46.541011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:46.541011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:46.541011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:46.541011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:46.541011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:46.541011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:46.541011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:46.541011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:46.541011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:46.541011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:46.541011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:07:46.541011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:07:46.541011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 48/99] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:07:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:07:46.546291 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c3f57-1bb5-4fb6-9ada-88437e49d759 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:46.546291 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c3f57-1bb5-4fb6-9ada-88437e49d759 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:46.546291 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c3f57-1bb5-4fb6-9ada-88437e49d759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:46.564402 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-474c3f57-1bb5-4fb6-9ada-88437e49d759 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:46.565692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-474c3f57-1bb5-4fb6-9ada-88437e49d759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:46.612517 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7e9b1-c30e-4adb-9f79-9096e285f688 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:46.626524 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:46.641005 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 49/100] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:46.642257 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c3f57-1bb5-4fb6-9ada-88437e49d759 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:46.657469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfd999-daea-4dd3-a1d0-3e5a60c5154f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:46.658442 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfd999-daea-4dd3-a1d0-3e5a60c5154f None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:46.658894 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfd999-daea-4dd3-a1d0-3e5a60c5154f None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:46.659540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfd999-daea-4dd3-a1d0-3e5a60c5154f None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:46.660472 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfd999-daea-4dd3-a1d0-3e5a60c5154f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:46.662419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfd999-daea-4dd3-a1d0-3e5a60c5154f None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:46.700594 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 52/101] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Oct 21 21:07:46 2022] POST /identity/v3/projects => generated 302 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:07:46.702815 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfd999-daea-4dd3-a1d0-3e5a60c5154f None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:46.768503 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c3f57-1bb5-4fb6-9ada-88437e49d759 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:46.785881 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:46.811774 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab80d64-1ddc-4585-9a4b-c46167de1f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:46.812905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab80d64-1ddc-4585-9a4b-c46167de1f49 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:46.813413 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab80d64-1ddc-4585-9a4b-c46167de1f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:46.813885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab80d64-1ddc-4585-9a4b-c46167de1f49 None admin] PATH_INFO: `/v3/users/63c48b9170d54c67a6f3f1ca3945291a` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:46.829658 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab80d64-1ddc-4585-9a4b-c46167de1f49 None admin] RBAC: Authorizing `identity:get_user(user_id=63c48b9170d54c67a6f3f1ca3945291a)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:46.831656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab80d64-1ddc-4585-9a4b-c46167de1f49 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:46.833851 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 50/102] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:46 2022] GET /identity/v3/users/63c48b9170d54c67a6f3f1ca3945291a => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:46.835555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab80d64-1ddc-4585-9a4b-c46167de1f49 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:46.846427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfd999-daea-4dd3-a1d0-3e5a60c5154f None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:46.893457 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bafe4-ebfa-48dc-9725-b50290463f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:46.894491 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bafe4-ebfa-48dc-9725-b50290463f9a None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:46.894970 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bafe4-ebfa-48dc-9725-b50290463f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:46.895465 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bafe4-ebfa-48dc-9725-b50290463f9a None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:46.896369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4bafe4-ebfa-48dc-9725-b50290463f9a None admin] RBAC: Adding query filter params (name=member) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:46.896958 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4bafe4-ebfa-48dc-9725-b50290463f9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:46.898806 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4bafe4-ebfa-48dc-9725-b50290463f9a None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:46.908041 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 53/103] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:07:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:46.911484 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53424f70-0cca-4f65-af53-35c168a1b079 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:46.912007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53424f70-0cca-4f65-af53-35c168a1b079 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:46.913040 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53424f70-0cca-4f65-af53-35c168a1b079 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:46.914504 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 54/104] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:46.915518 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53424f70-0cca-4f65-af53-35c168a1b079 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:46.952798 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:46.958979 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab80d64-1ddc-4585-9a4b-c46167de1f49 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:46.999030 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f9e718-b855-48cc-8887-f4969272f28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:46.999610 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9e718-b855-48cc-8887-f4969272f28d None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:46.999904 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9e718-b855-48cc-8887-f4969272f28d None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:47.000184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9e718-b855-48cc-8887-f4969272f28d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:47.010765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9e718-b855-48cc-8887-f4969272f28d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:47.015900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-81f9e718-b855-48cc-8887-f4969272f28d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:07:47.015900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:47.015900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:47.015900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:47.015900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:47.015900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:47.015900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:47.015900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:47.015900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:47.015900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:47.015900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:47.015900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:47.015900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:47.015900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:47.015900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:47.015900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:47.016718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:47.016718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:47.016718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:47.016718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:47.016718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:47.016718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:47.016718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:47.016718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:47.016718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:47.016718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:47.016718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:47.016718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:47.016718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:47.016718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:47.016718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:47.016718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:47.017458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:47.017458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:47.017458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:47.017458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:47.017458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:47.017458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:47.017458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:47.017458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:47.017458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:47.017458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:47.017458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:07:47.017458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:07:47.017458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 51/105] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:07:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:07:47.019995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b5de0-22a0-4936-8ff0-ee2643767031 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:47.020288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b5de0-22a0-4936-8ff0-ee2643767031 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:47.020566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b5de0-22a0-4936-8ff0-ee2643767031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:47.039006 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-8e6b5de0-22a0-4936-8ff0-ee2643767031 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:47.040209 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8e6b5de0-22a0-4936-8ff0-ee2643767031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:47.042230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53424f70-0cca-4f65-af53-35c168a1b079 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:47.066881 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:47.079539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa19e92-cc1f-4d7b-8b8a-8c1f9a5fa0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:47.080206 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa19e92-cc1f-4d7b-8b8a-8c1f9a5fa0f7 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:47.080489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa19e92-cc1f-4d7b-8b8a-8c1f9a5fa0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:47.080917 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa19e92-cc1f-4d7b-8b8a-8c1f9a5fa0f7 None admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:47.082046 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa19e92-cc1f-4d7b-8b8a-8c1f9a5fa0f7 None admin] RBAC: Authorizing `identity:get_project(project_id=dd6450bb3add4920a802300916dc915c)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:47.083485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa19e92-cc1f-4d7b-8b8a-8c1f9a5fa0f7 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:47.085182 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 55/106] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:07:46 2022] GET /identity/v3/projects/dd6450bb3add4920a802300916dc915c => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:47.085941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa19e92-cc1f-4d7b-8b8a-8c1f9a5fa0f7 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:47.115485 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 52/107] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:47.123787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b5de0-22a0-4936-8ff0-ee2643767031 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:47.193367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:47.202099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa19e92-cc1f-4d7b-8b8a-8c1f9a5fa0f7 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:47.230788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b5de0-22a0-4936-8ff0-ee2643767031 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:47.244748 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83e2fb-b88a-4339-8af2-72a5e66c495b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:47.245795 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83e2fb-b88a-4339-8af2-72a5e66c495b None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:47.246086 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83e2fb-b88a-4339-8af2-72a5e66c495b None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:47.246413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83e2fb-b88a-4339-8af2-72a5e66c495b None admin] PATH_INFO: `/v3/users/8e8963c7ee674a639acccd9a093fe9ee` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:47.247899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83e2fb-b88a-4339-8af2-72a5e66c495b None admin] RBAC: Authorizing `identity:get_user(user_id=8e8963c7ee674a639acccd9a093fe9ee)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:47.249883 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83e2fb-b88a-4339-8af2-72a5e66c495b None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:47.251814 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 56/108] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:47 2022] GET /identity/v3/users/8e8963c7ee674a639acccd9a093fe9ee => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:47.253412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83e2fb-b88a-4339-8af2-72a5e66c495b None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:47.266535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35e126-db57-4307-835d-dcd344d273c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:47.267322 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35e126-db57-4307-835d-dcd344d273c7 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:47.267746 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35e126-db57-4307-835d-dcd344d273c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:47.268117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35e126-db57-4307-835d-dcd344d273c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:47.280042 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35e126-db57-4307-835d-dcd344d273c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:47.285848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-6d35e126-db57-4307-835d-dcd344d273c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:07:47.285848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:47.285848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:47.285848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:47.285848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:47.285848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:47.285848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:47.285848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:47.285848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:47.285848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:47.285848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:47.285848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:47.285848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:47.285848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:47.285848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:47.285848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:47.286692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:47.286692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:47.286692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:47.286692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:47.286692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:47.286692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:47.286692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:47.286692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:47.286692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:47.286692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:47.286692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:47.286692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:47.286692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:47.286692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:47.286692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:47.286692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:47.288781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:47.288781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:47.288781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:47.288781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:47.288781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:47.288781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:47.288781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:47.288781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:47.288781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:47.288781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:47.288781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:07:47.288781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:07:47.288781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 53/109] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Oct 21 21:07:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:07:47.290460 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35e126-db57-4307-835d-dcd344d273c7 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:47.314868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:47.380532 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83e2fb-b88a-4339-8af2-72a5e66c495b None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:47.393935 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35e126-db57-4307-835d-dcd344d273c7 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:47.416169 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6a971-85c4-44b9-9f02-5513ac85a006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:47.416786 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6a971-85c4-44b9-9f02-5513ac85a006 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:47.417200 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6a971-85c4-44b9-9f02-5513ac85a006 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:47.417384 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6a971-85c4-44b9-9f02-5513ac85a006 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:47.418056 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6a971-85c4-44b9-9f02-5513ac85a006 None admin] RBAC: Adding query filter params (name=member) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:47.418357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6a971-85c4-44b9-9f02-5513ac85a006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:47.419744 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6a971-85c4-44b9-9f02-5513ac85a006 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:47.428931 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 57/110] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:07:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:47.429576 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6a971-85c4-44b9-9f02-5513ac85a006 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:47.430938 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cef04-acd5-4dda-a263-3645254c506b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:47.431784 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cef04-acd5-4dda-a263-3645254c506b None admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:47.432222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cef04-acd5-4dda-a263-3645254c506b None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:47.432630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cef04-acd5-4dda-a263-3645254c506b None admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c/users/63c48b9170d54c67a6f3f1ca3945291a/roles/c6ec89ead9d24571b777018f6a533293` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:47.434698 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cef04-acd5-4dda-a263-3645254c506b None admin] RBAC: Authorizing `identity:create_grant(project_id=dd6450bb3add4920a802300916dc915c, user_id=63c48b9170d54c67a6f3f1ca3945291a, role_id=c6ec89ead9d24571b777018f6a533293)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:47.436384 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cef04-acd5-4dda-a263-3645254c506b None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:47.453592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 54/111] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:07:47 2022] PUT /identity/v3/projects/dd6450bb3add4920a802300916dc915c/users/63c48b9170d54c67a6f3f1ca3945291a/roles/c6ec89ead9d24571b777018f6a533293 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:07:47.455056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cef04-acd5-4dda-a263-3645254c506b None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:47.462508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:47.560022 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:47.582432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cef04-acd5-4dda-a263-3645254c506b None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:47.582668 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6a971-85c4-44b9-9f02-5513ac85a006 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:47.621478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780445f8-44d1-4567-9e37-8465ef12a5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:47.622218 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780445f8-44d1-4567-9e37-8465ef12a5a7 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:47.622561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780445f8-44d1-4567-9e37-8465ef12a5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:47.622841 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780445f8-44d1-4567-9e37-8465ef12a5a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:47.623683 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780445f8-44d1-4567-9e37-8465ef12a5a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:47.623996 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780445f8-44d1-4567-9e37-8465ef12a5a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:47.625406 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780445f8-44d1-4567-9e37-8465ef12a5a7 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:47.629917 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51065717-1d0d-4b09-9490-f6b54631e982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:47.631755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51065717-1d0d-4b09-9490-f6b54631e982 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:47.632094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51065717-1d0d-4b09-9490-f6b54631e982 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:47.632974 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51065717-1d0d-4b09-9490-f6b54631e982 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:47.635912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 55/112] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:07:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:47.637236 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780445f8-44d1-4567-9e37-8465ef12a5a7 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:47.713074 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51065717-1d0d-4b09-9490-f6b54631e982 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:47.714555 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51065717-1d0d-4b09-9490-f6b54631e982 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:47.739020 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:47.757361 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780445f8-44d1-4567-9e37-8465ef12a5a7 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:47.792437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69837927-2446-4aa7-827a-c89e7466a7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:47.793337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69837927-2446-4aa7-827a-c89e7466a7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:47.793755 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69837927-2446-4aa7-827a-c89e7466a7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:47.794269 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69837927-2446-4aa7-827a-c89e7466a7e6 None admin] PATH_INFO: `/v3/users/8e8963c7ee674a639acccd9a093fe9ee` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:47.795545 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69837927-2446-4aa7-827a-c89e7466a7e6 None admin] RBAC: Authorizing `identity:get_user(user_id=8e8963c7ee674a639acccd9a093fe9ee)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:47.797024 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69837927-2446-4aa7-827a-c89e7466a7e6 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:47.798659 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 56/113] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:47 2022] GET /identity/v3/users/8e8963c7ee674a639acccd9a093fe9ee => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:47.803152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69837927-2446-4aa7-827a-c89e7466a7e6 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:47.814542 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:47.821533 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 58/114] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:07:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:07:47.822693 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51065717-1d0d-4b09-9490-f6b54631e982 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:47.904803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69837927-2446-4aa7-827a-c89e7466a7e6 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:47.923076 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51065717-1d0d-4b09-9490-f6b54631e982 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:47.938601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055465e-72c4-4ca2-ac11-5c8c8bd6134c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:47.939234 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055465e-72c4-4ca2-ac11-5c8c8bd6134c None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:47.939513 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055465e-72c4-4ca2-ac11-5c8c8bd6134c None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:47.939815 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055465e-72c4-4ca2-ac11-5c8c8bd6134c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:47.948089 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:47.953617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055465e-72c4-4ca2-ac11-5c8c8bd6134c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:47.967591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-8055465e-72c4-4ca2-ac11-5c8c8bd6134c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:07:47.967591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:47.967591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:47.967591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:47.967591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:47.967591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:47.967591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:47.967591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:47.967591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:47.967591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:47.967591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:47.967591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:47.967591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:47.967591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:47.967591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:47.967591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:47.968473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:47.968473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:47.968473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:47.968473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:47.968473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:47.968473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:47.968473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:47.968473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:47.968473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:47.968473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:47.968473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:47.968473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:47.968473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:47.968473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:47.968473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:47.968473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:47.969200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:47.969200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:47.969200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:47.969200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:47.969200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:47.969200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:47.969200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:47.969200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:47.969200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:47.969200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:47.969200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:07:47.969200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:07:47.969200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 57/115] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:07:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:07:47.969200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055465e-72c4-4ca2-ac11-5c8c8bd6134c None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:47.969859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb89662-a0a3-4f29-8077-9d6868727b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:47.969859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb89662-a0a3-4f29-8077-9d6868727b32 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:47.969859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb89662-a0a3-4f29-8077-9d6868727b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:47.969859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb89662-a0a3-4f29-8077-9d6868727b32 None admin] PATH_INFO: `/v3/users/8e8963c7ee674a639acccd9a093fe9ee` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:47.970095 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb89662-a0a3-4f29-8077-9d6868727b32 None admin] RBAC: Authorizing `identity:get_user(user_id=8e8963c7ee674a639acccd9a093fe9ee)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:47.970095 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb89662-a0a3-4f29-8077-9d6868727b32 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:47.970095 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 59/116] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:47 2022] GET /identity/v3/users/8e8963c7ee674a639acccd9a093fe9ee => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:47.970095 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb89662-a0a3-4f29-8077-9d6868727b32 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:48.042005 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:48.081184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055465e-72c4-4ca2-ac11-5c8c8bd6134c None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:48.108225 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb89662-a0a3-4f29-8077-9d6868727b32 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:48.120952 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2443d7-5d97-4f26-9755-1fb8ef6a785f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:48.122093 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2443d7-5d97-4f26-9755-1fb8ef6a785f None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:48.122616 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2443d7-5d97-4f26-9755-1fb8ef6a785f None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:48.123519 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2443d7-5d97-4f26-9755-1fb8ef6a785f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:48.147311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851b0a7-f649-43af-a516-6f1dcab8f727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:48.148647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851b0a7-f649-43af-a516-6f1dcab8f727 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:48.149542 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851b0a7-f649-43af-a516-6f1dcab8f727 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:48.150040 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851b0a7-f649-43af-a516-6f1dcab8f727 None admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:48.151443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851b0a7-f649-43af-a516-6f1dcab8f727 None admin] RBAC: Authorizing `identity:get_project(project_id=dd6450bb3add4920a802300916dc915c)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:48.153918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851b0a7-f649-43af-a516-6f1dcab8f727 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:48.156872 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 60/117] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:07:47 2022] GET /identity/v3/projects/dd6450bb3add4920a802300916dc915c => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:48.158100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851b0a7-f649-43af-a516-6f1dcab8f727 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:48.203597 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2443d7-5d97-4f26-9755-1fb8ef6a785f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:48.204758 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2443d7-5d97-4f26-9755-1fb8ef6a785f None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:48.229936 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:48.275710 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851b0a7-f649-43af-a516-6f1dcab8f727 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:48.279273 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:48.313583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 58/118] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:07:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:07:48.314434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2443d7-5d97-4f26-9755-1fb8ef6a785f None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:48.315815 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c45326-75cd-40eb-940f-b1f3980f7897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:48.316788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c45326-75cd-40eb-940f-b1f3980f7897 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:48.317402 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c45326-75cd-40eb-940f-b1f3980f7897 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:48.317873 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c45326-75cd-40eb-940f-b1f3980f7897 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:48.319435 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c45326-75cd-40eb-940f-b1f3980f7897 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:48.319980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c45326-75cd-40eb-940f-b1f3980f7897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:48.321542 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c45326-75cd-40eb-940f-b1f3980f7897 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:48.330850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 61/119] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:07:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:48.332015 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c45326-75cd-40eb-940f-b1f3980f7897 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:48.424366 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2443d7-5d97-4f26-9755-1fb8ef6a785f None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:48.443414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c45326-75cd-40eb-940f-b1f3980f7897 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:48.459979 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94e2ee-322b-4757-b16d-894ef6d8a3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:48.460595 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94e2ee-322b-4757-b16d-894ef6d8a3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:48.460878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94e2ee-322b-4757-b16d-894ef6d8a3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:48.461142 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94e2ee-322b-4757-b16d-894ef6d8a3b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:48.469491 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:48.482938 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a95bbc-495b-48bb-9f6e-1599dc180293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:48.483903 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a95bbc-495b-48bb-9f6e-1599dc180293 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:48.484384 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a95bbc-495b-48bb-9f6e-1599dc180293 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:48.484951 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a95bbc-495b-48bb-9f6e-1599dc180293 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:48.485932 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a95bbc-495b-48bb-9f6e-1599dc180293 None admin] RBAC: Adding query filter params (role.id=c73d9ac0c74147e095a9a038d200e2e9, scope.project.id=dd6450bb3add4920a802300916dc915c, user.id=8e8963c7ee674a639acccd9a093fe9ee) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:48.486531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a95bbc-495b-48bb-9f6e-1599dc180293 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:48.488344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a95bbc-495b-48bb-9f6e-1599dc180293 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:48.499950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 62/120] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:07:48 2022] GET /identity/v3/role_assignments?user.id=8e8963c7ee674a639acccd9a093fe9ee&scope.project.id=dd6450bb3add4920a802300916dc915c&role.id=c73d9ac0c74147e095a9a038d200e2e9 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:48.500766 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a95bbc-495b-48bb-9f6e-1599dc180293 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:48.509050 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:48.541023 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94e2ee-322b-4757-b16d-894ef6d8a3b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:48.542602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94e2ee-322b-4757-b16d-894ef6d8a3b3 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:48.642146 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a95bbc-495b-48bb-9f6e-1599dc180293 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:48.686231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff862d5-3c72-4e8b-915f-4fa1906b2cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:48.687296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff862d5-3c72-4e8b-915f-4fa1906b2cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:48.687819 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff862d5-3c72-4e8b-915f-4fa1906b2cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:48.688434 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff862d5-3c72-4e8b-915f-4fa1906b2cf5 None admin] PATH_INFO: `/v3/projects/9dfec435f3e7456aa2d5b896994cdec4` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:48.689861 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff862d5-3c72-4e8b-915f-4fa1906b2cf5 None admin] RBAC: Authorizing `identity:get_project(project_id=9dfec435f3e7456aa2d5b896994cdec4)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:48.692426 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff862d5-3c72-4e8b-915f-4fa1906b2cf5 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:48.694447 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 63/121] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:07:48 2022] GET /identity/v3/projects/9dfec435f3e7456aa2d5b896994cdec4 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:48.695438 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff862d5-3c72-4e8b-915f-4fa1906b2cf5 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:48.748294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 59/122] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:07:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:07:48.749222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94e2ee-322b-4757-b16d-894ef6d8a3b3 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:48.760612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:48.808518 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff862d5-3c72-4e8b-915f-4fa1906b2cf5 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:48.830907 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:48.844860 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5d669-c900-4ba0-a0ee-e5e267352152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:48.845847 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5d669-c900-4ba0-a0ee-e5e267352152 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:48.846348 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5d669-c900-4ba0-a0ee-e5e267352152 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:48.846818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5d669-c900-4ba0-a0ee-e5e267352152 None admin] PATH_INFO: `/v3/users/85a2a3bc91ec4aebbb1e136167a1d87f` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:48.848184 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd5d669-c900-4ba0-a0ee-e5e267352152 None admin] RBAC: Authorizing `identity:get_user(user_id=85a2a3bc91ec4aebbb1e136167a1d87f)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:48.849922 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd5d669-c900-4ba0-a0ee-e5e267352152 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:48.851811 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 64/123] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:48 2022] GET /identity/v3/users/85a2a3bc91ec4aebbb1e136167a1d87f => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:48.852777 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5d669-c900-4ba0-a0ee-e5e267352152 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:48.868775 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94e2ee-322b-4757-b16d-894ef6d8a3b3 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:48.908192 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56330956-6033-4667-9549-8803bef651d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:48.908747 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56330956-6033-4667-9549-8803bef651d5 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:48.909044 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56330956-6033-4667-9549-8803bef651d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:48.909301 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56330956-6033-4667-9549-8803bef651d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:48.910042 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56330956-6033-4667-9549-8803bef651d5 None admin] RBAC: Adding query filter params (role.id=c73d9ac0c74147e095a9a038d200e2e9, scope.project.id=9dfec435f3e7456aa2d5b896994cdec4, user.id=8e8963c7ee674a639acccd9a093fe9ee) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:48.910354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56330956-6033-4667-9549-8803bef651d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:48.912201 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56330956-6033-4667-9549-8803bef651d5 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:48.918739 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 60/124] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:07:48 2022] GET /identity/v3/role_assignments?user.id=8e8963c7ee674a639acccd9a093fe9ee&scope.project.id=9dfec435f3e7456aa2d5b896994cdec4&role.id=c73d9ac0c74147e095a9a038d200e2e9 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:48.919716 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56330956-6033-4667-9549-8803bef651d5 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:48.973252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5d669-c900-4ba0-a0ee-e5e267352152 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:49.008100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:49.020267 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffe3d4-c98a-41b8-92a2-43252d975b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:49.021231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffe3d4-c98a-41b8-92a2-43252d975b41 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:49.021519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffe3d4-c98a-41b8-92a2-43252d975b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:49.021831 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffe3d4-c98a-41b8-92a2-43252d975b41 None admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:49.024447 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffe3d4-c98a-41b8-92a2-43252d975b41 None admin] RBAC: Authorizing `identity:get_project(project_id=dd6450bb3add4920a802300916dc915c)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:49.025968 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffe3d4-c98a-41b8-92a2-43252d975b41 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:49.027873 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 65/125] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:07:48 2022] GET /identity/v3/projects/dd6450bb3add4920a802300916dc915c => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:49.033779 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffe3d4-c98a-41b8-92a2-43252d975b41 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:49.051137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56330956-6033-4667-9549-8803bef651d5 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:49.125409 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a0e3d-6eb8-48e8-b081-857a47adfb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:49.126054 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a0e3d-6eb8-48e8-b081-857a47adfb63 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:49.126338 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a0e3d-6eb8-48e8-b081-857a47adfb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:49.126610 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a0e3d-6eb8-48e8-b081-857a47adfb63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:49.134989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:49.154879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffe3d4-c98a-41b8-92a2-43252d975b41 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:49.200175 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c959b2-540a-4e60-ac01-7c4c57fcaf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:49.201584 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c959b2-540a-4e60-ac01-7c4c57fcaf6a None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:49.202333 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c959b2-540a-4e60-ac01-7c4c57fcaf6a None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:49.202803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c959b2-540a-4e60-ac01-7c4c57fcaf6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:49.203899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c959b2-540a-4e60-ac01-7c4c57fcaf6a None admin] RBAC: Adding query filter params (role.id=01dbc90c2ce041afa161e621bd524341, scope.project.id=dd6450bb3add4920a802300916dc915c, user.id=8e8963c7ee674a639acccd9a093fe9ee) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:49.204693 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c959b2-540a-4e60-ac01-7c4c57fcaf6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:49.206278 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c959b2-540a-4e60-ac01-7c4c57fcaf6a None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:49.213634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 66/126] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:07:49 2022] GET /identity/v3/role_assignments?user.id=8e8963c7ee674a639acccd9a093fe9ee&scope.project.id=dd6450bb3add4920a802300916dc915c&role.id=01dbc90c2ce041afa161e621bd524341 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:49.214714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9ffbc-4216-4fd8-9657-96bc508df986 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:49.215004 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9ffbc-4216-4fd8-9657-96bc508df986 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:49.215273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9ffbc-4216-4fd8-9657-96bc508df986 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:49.216676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 67/127] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:49.222752 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a0e3d-6eb8-48e8-b081-857a47adfb63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:49.223033 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aed0b4-0881-481c-88b5-5b819255fdfd None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:49.223033 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aed0b4-0881-481c-88b5-5b819255fdfd None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:49.223312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aed0b4-0881-481c-88b5-5b819255fdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:49.224307 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a0e3d-6eb8-48e8-b081-857a47adfb63 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:49.242178 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-e3aed0b4-0881-481c-88b5-5b819255fdfd None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:49.243352 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-e3aed0b4-0881-481c-88b5-5b819255fdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:49.300261 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:49.320994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 68/128] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:07:49.351008 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 61/129] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:07:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:07:49.357058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aed0b4-0881-481c-88b5-5b819255fdfd None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:49.468831 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aed0b4-0881-481c-88b5-5b819255fdfd None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:49.504643 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94433c55-cdc2-4f8e-84c6-70dbb9722e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:49.505623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94433c55-cdc2-4f8e-84c6-70dbb9722e76 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:49.506179 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94433c55-cdc2-4f8e-84c6-70dbb9722e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:49.507141 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94433c55-cdc2-4f8e-84c6-70dbb9722e76 None admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:49.508536 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94433c55-cdc2-4f8e-84c6-70dbb9722e76 None admin] RBAC: Authorizing `identity:get_project(project_id=dd6450bb3add4920a802300916dc915c)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:49.510190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94433c55-cdc2-4f8e-84c6-70dbb9722e76 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:49.512326 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 69/130] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:07:49 2022] GET /identity/v3/projects/dd6450bb3add4920a802300916dc915c => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:49.518308 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a0e3d-6eb8-48e8-b081-857a47adfb63 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:49.554194 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:49.618345 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a0e3d-6eb8-48e8-b081-857a47adfb63 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:49.622221 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94433c55-cdc2-4f8e-84c6-70dbb9722e76 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:49.654081 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7bbf-27c3-4d8b-964a-4ccf2e2a1079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:49.654694 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7bbf-27c3-4d8b-964a-4ccf2e2a1079 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:49.654989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7bbf-27c3-4d8b-964a-4ccf2e2a1079 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:49.655260 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7bbf-27c3-4d8b-964a-4ccf2e2a1079 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:49.655994 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e7bbf-27c3-4d8b-964a-4ccf2e2a1079 None admin] RBAC: Adding query filter params (role.id=c514b065b7604094b899c586f79b0c5b, scope.project.id=dd6450bb3add4920a802300916dc915c, user.id=85a2a3bc91ec4aebbb1e136167a1d87f) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:49.656311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e7bbf-27c3-4d8b-964a-4ccf2e2a1079 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:49.657768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e7bbf-27c3-4d8b-964a-4ccf2e2a1079 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:49.663805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 62/131] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:07:49 2022] GET /identity/v3/role_assignments?user.id=85a2a3bc91ec4aebbb1e136167a1d87f&scope.project.id=dd6450bb3add4920a802300916dc915c&role.id=c514b065b7604094b899c586f79b0c5b => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:07:49.681923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:49.741133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94433c55-cdc2-4f8e-84c6-70dbb9722e76 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:49.782802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562749a-22a3-41fe-81a1-04859e9fcc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:49.783530 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562749a-22a3-41fe-81a1-04859e9fcc1b None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:49.783843 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562749a-22a3-41fe-81a1-04859e9fcc1b None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:49.784132 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562749a-22a3-41fe-81a1-04859e9fcc1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:49.785332 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562749a-22a3-41fe-81a1-04859e9fcc1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:49.786827 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562749a-22a3-41fe-81a1-04859e9fcc1b None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:49.789256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 70/132] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:07:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:07:49.794703 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7bbf-27c3-4d8b-964a-4ccf2e2a1079 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:49.879901 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b3c25-4a68-4bf5-850d-4b0cd7960a5a None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:49.880195 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b3c25-4a68-4bf5-850d-4b0cd7960a5a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:49.880453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b3c25-4a68-4bf5-850d-4b0cd7960a5a None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:49.882569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 71/133] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:49.887551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e76f75-6fe4-4554-9a81-dbbd62cce62e None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:49.887886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e76f75-6fe4-4554-9a81-dbbd62cce62e None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:49.888613 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e76f75-6fe4-4554-9a81-dbbd62cce62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:49.900713 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:49.903652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7bbf-27c3-4d8b-964a-4ccf2e2a1079 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:49.912857 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-d7e76f75-6fe4-4554-9a81-dbbd62cce62e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:49.914061 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d7e76f75-6fe4-4554-9a81-dbbd62cce62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:49.936297 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcaf79-f307-4f85-8171-d4df994af92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:49.936907 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcaf79-f307-4f85-8171-d4df994af92c None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:49.937185 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcaf79-f307-4f85-8171-d4df994af92c None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:49.937461 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcaf79-f307-4f85-8171-d4df994af92c None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:49.938113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dcaf79-f307-4f85-8171-d4df994af92c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:49.939591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dcaf79-f307-4f85-8171-d4df994af92c None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:49.962167 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 63/134] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:07:49 2022] POST /identity/v3/users => generated 283 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:07:49.981675 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 72/135] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:07:50.249741 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcaf79-f307-4f85-8171-d4df994af92c None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:50.357208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcaf79-f307-4f85-8171-d4df994af92c None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:50.393825 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9278d-7aa6-49a0-81e1-695273a99038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:50.394471 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9278d-7aa6-49a0-81e1-695273a99038 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:50.394755 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9278d-7aa6-49a0-81e1-695273a99038 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:50.395032 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9278d-7aa6-49a0-81e1-695273a99038 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:50.406634 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9278d-7aa6-49a0-81e1-695273a99038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:50.411980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-28b9278d-7aa6-49a0-81e1-695273a99038 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 21:07:50.411980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:50.411980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:50.411980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:50.411980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:50.411980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:50.411980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:50.411980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:50.411980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:50.411980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:50.411980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:50.411980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:50.411980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:50.411980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:50.411980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:50.411980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:50.412885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:50.412885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:50.412885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:50.412885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:50.412885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:50.412885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:50.412885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:50.412885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:50.412885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:50.412885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:50.412885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:50.412885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:50.412885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:50.412885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:50.412885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:50.412885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:50.413627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:50.413627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:50.413627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:50.413627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:50.413627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:50.413627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:50.413627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:50.413627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:50.413627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:50.413627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:50.413627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 21:07:50.413627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:07:50.413627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 64/136] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:07:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:07:50.419350 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e76f75-6fe4-4554-9a81-dbbd62cce62e None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:50.476302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:50.531153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e76f75-6fe4-4554-9a81-dbbd62cce62e None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:50.567415 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9637860-640e-4dcf-9114-1ce52862a6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:50.568481 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9637860-640e-4dcf-9114-1ce52862a6ce None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:50.568989 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9637860-640e-4dcf-9114-1ce52862a6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:50.569457 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9637860-640e-4dcf-9114-1ce52862a6ce None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:50.570313 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9637860-640e-4dcf-9114-1ce52862a6ce None admin] RBAC: Adding query filter params (name=reader) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:50.570836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9637860-640e-4dcf-9114-1ce52862a6ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:50.572421 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9637860-640e-4dcf-9114-1ce52862a6ce None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:50.581402 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 73/137] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:07:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:50.587799 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9278d-7aa6-49a0-81e1-695273a99038 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:50.606973 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:50.695868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9278d-7aa6-49a0-81e1-695273a99038 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:50.733348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6a143-c05f-41ee-a8f4-bd918c725ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:50.734326 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6a143-c05f-41ee-a8f4-bd918c725ebe None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:50.734804 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6a143-c05f-41ee-a8f4-bd918c725ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:50.735275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6a143-c05f-41ee-a8f4-bd918c725ebe None admin] PATH_INFO: `/v3/users/63c48b9170d54c67a6f3f1ca3945291a` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:50.739605 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6a143-c05f-41ee-a8f4-bd918c725ebe None admin] RBAC: Authorizing `identity:get_user(user_id=63c48b9170d54c67a6f3f1ca3945291a)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:50.739605 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6a143-c05f-41ee-a8f4-bd918c725ebe None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:50.740138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 65/138] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:50 2022] GET /identity/v3/users/63c48b9170d54c67a6f3f1ca3945291a => generated 289 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:50.748709 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9637860-640e-4dcf-9114-1ce52862a6ce None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:50.855359 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9637860-640e-4dcf-9114-1ce52862a6ce None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:50.869008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:50.893437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3f6e0-e629-4be7-b0ea-a5cf7db257e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:50.894395 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3f6e0-e629-4be7-b0ea-a5cf7db257e4 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:50.894878 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3f6e0-e629-4be7-b0ea-a5cf7db257e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:50.895350 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3f6e0-e629-4be7-b0ea-a5cf7db257e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:50.969378 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3f6e0-e629-4be7-b0ea-a5cf7db257e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:50.971182 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3f6e0-e629-4be7-b0ea-a5cf7db257e4 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:51.074147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 74/139] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:07:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:07:51.080320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6a143-c05f-41ee-a8f4-bd918c725ebe None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:51.168462 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:51.186513 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6a143-c05f-41ee-a8f4-bd918c725ebe None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:51.220463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf252bc-8ec5-4632-a15a-17d2807844ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:51.221372 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf252bc-8ec5-4632-a15a-17d2807844ff None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:51.221855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf252bc-8ec5-4632-a15a-17d2807844ff None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:51.222347 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf252bc-8ec5-4632-a15a-17d2807844ff None admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:51.223780 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf252bc-8ec5-4632-a15a-17d2807844ff None admin] RBAC: Authorizing `identity:get_project(project_id=dd6450bb3add4920a802300916dc915c)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:51.226146 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf252bc-8ec5-4632-a15a-17d2807844ff None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:51.228070 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 66/140] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:07:51 2022] GET /identity/v3/projects/dd6450bb3add4920a802300916dc915c => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:51.236656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3f6e0-e629-4be7-b0ea-a5cf7db257e4 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:51.253046 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:51.343498 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3f6e0-e629-4be7-b0ea-a5cf7db257e4 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:51.378245 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be79ff-6917-4056-a4df-8ed5cc17dcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:51.379177 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be79ff-6917-4056-a4df-8ed5cc17dcf8 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:51.379716 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be79ff-6917-4056-a4df-8ed5cc17dcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:51.380201 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be79ff-6917-4056-a4df-8ed5cc17dcf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:51.381648 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be79ff-6917-4056-a4df-8ed5cc17dcf8 None admin] RBAC: Adding query filter params (role.id=c6ec89ead9d24571b777018f6a533293, scope.project.id=dd6450bb3add4920a802300916dc915c, user.id=63c48b9170d54c67a6f3f1ca3945291a) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:51.382175 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be79ff-6917-4056-a4df-8ed5cc17dcf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:51.383854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be79ff-6917-4056-a4df-8ed5cc17dcf8 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:51.389868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 75/141] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:07:51 2022] GET /identity/v3/role_assignments?user.id=63c48b9170d54c67a6f3f1ca3945291a&scope.project.id=dd6450bb3add4920a802300916dc915c&role.id=c6ec89ead9d24571b777018f6a533293 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:07:52.226355 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1304d83-4e3a-4196-8481-3d3bd7c23d3e None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:52.226887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1304d83-4e3a-4196-8481-3d3bd7c23d3e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:52.226887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1304d83-4e3a-4196-8481-3d3bd7c23d3e None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:52.228434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 67/142] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:07:52.233487 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb3aa1-2341-49cd-8d0e-9e3e4f8cf197 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:52.233862 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb3aa1-2341-49cd-8d0e-9e3e4f8cf197 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:52.234033 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb3aa1-2341-49cd-8d0e-9e3e4f8cf197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:52.254477 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-c0fb3aa1-2341-49cd-8d0e-9e3e4f8cf197 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:52.256520 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c0fb3aa1-2341-49cd-8d0e-9e3e4f8cf197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:52.326451 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 76/143] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:52.366282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f415dee-4359-43e3-a4b0-47e4f7b7fcc6 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:52.366613 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f415dee-4359-43e3-a4b0-47e4f7b7fcc6 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:52.366848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f415dee-4359-43e3-a4b0-47e4f7b7fcc6 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:52.368234 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 68/144] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:52.373343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae20726-637d-406d-90bd-3a5661569cab None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:52.373635 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae20726-637d-406d-90bd-3a5661569cab None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:52.373913 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae20726-637d-406d-90bd-3a5661569cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:52.386646 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:52.393640 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-7ae20726-637d-406d-90bd-3a5661569cab None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:52.394830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-7ae20726-637d-406d-90bd-3a5661569cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:52.464898 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 77/145] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:52.551733 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41e293-07c4-40d6-b7a9-3586bbb598de None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:52.551938 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41e293-07c4-40d6-b7a9-3586bbb598de None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:52.552369 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41e293-07c4-40d6-b7a9-3586bbb598de None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:52.554247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 69/146] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:52.559347 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0f09a-938d-4142-b5d6-64e39e973a67 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:52.559667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0f09a-938d-4142-b5d6-64e39e973a67 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:52.559941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0f09a-938d-4142-b5d6-64e39e973a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:52.577731 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-ddd0f09a-938d-4142-b5d6-64e39e973a67 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:52.578892 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ddd0f09a-938d-4142-b5d6-64e39e973a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:52.595377 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41e293-07c4-40d6-b7a9-3586bbb598de None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:52.646612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 78/147] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:52.695139 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41e293-07c4-40d6-b7a9-3586bbb598de None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:52.729689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0f09a-938d-4142-b5d6-64e39e973a67 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:52.731890 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45103aec-e3a1-468e-9b9c-7298538b42dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:52.733786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45103aec-e3a1-468e-9b9c-7298538b42dd None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:52.733786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45103aec-e3a1-468e-9b9c-7298538b42dd None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:52.733786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45103aec-e3a1-468e-9b9c-7298538b42dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:52.745739 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45103aec-e3a1-468e-9b9c-7298538b42dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:52.751219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-45103aec-e3a1-468e-9b9c-7298538b42dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:07:52.751219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:52.751219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:52.751219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:52.751219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:52.751219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:52.751219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:52.751219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:52.751219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:52.751219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:52.751219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:52.751219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:52.751219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:52.751219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:52.751219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:52.751219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:52.752176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:52.752176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:52.752176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:52.752176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:52.752176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:52.752176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:52.752176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:52.752176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:52.752176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:52.752176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:52.752176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:52.752176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:52.752176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:52.752176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:52.752176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:52.752176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:52.753120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:52.753120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:52.753120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:52.753120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:52.753120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:52.753120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:52.753120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:52.753120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:52.753120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:52.753120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:52.753120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:07:52.753120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:07:52.755200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 70/148] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:07:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:07:52.761029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45103aec-e3a1-468e-9b9c-7298538b42dd None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:52.776185 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:52.805072 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:52.844395 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0f09a-938d-4142-b5d6-64e39e973a67 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:52.864259 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45103aec-e3a1-468e-9b9c-7298538b42dd None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:52.881709 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb76dc-b6b2-4ae7-b6d4-189b3dc17f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:52.882580 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb76dc-b6b2-4ae7-b6d4-189b3dc17f3d None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:52.883047 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb76dc-b6b2-4ae7-b6d4-189b3dc17f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:52.883524 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb76dc-b6b2-4ae7-b6d4-189b3dc17f3d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:52.894394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb76dc-b6b2-4ae7-b6d4-189b3dc17f3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:52.899267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08c847-386c-4d98-942a-e801f9be531c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:52.899911 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08c847-386c-4d98-942a-e801f9be531c None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:52.900193 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08c847-386c-4d98-942a-e801f9be531c None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:52.900468 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08c847-386c-4d98-942a-e801f9be531c None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:52.900624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-c7eb76dc-b6b2-4ae7-b6d4-189b3dc17f3d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:07:52.900624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:52.900624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:52.900624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:52.900624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:52.900624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:52.900624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:52.900624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:52.900624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:52.900624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:52.900624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:52.900624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:52.900624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:52.900624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:52.900624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:52.900624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:52.901431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:52.901431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:52.901431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:52.901431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:52.901431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:52.901431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:52.901431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:52.901431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:52.901431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:52.901431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:52.901431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:52.901431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:52.901431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:52.901431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:52.901431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:52.901431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:52.904331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:52.904331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:52.904331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:52.904331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:52.904331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:52.904331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:52.904331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:52.904331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:52.904331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:52.904331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:52.904331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:07:52.904331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:07:52.904938 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08c847-386c-4d98-942a-e801f9be531c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:52.904938 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08c847-386c-4d98-942a-e801f9be531c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:52.905053 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 79/149] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Oct 21 21:07:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:07:52.905199 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08c847-386c-4d98-942a-e801f9be531c None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:52.907220 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb76dc-b6b2-4ae7-b6d4-189b3dc17f3d None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:52.915066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 71/150] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:07:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:52.918610 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08c847-386c-4d98-942a-e801f9be531c None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:52.991948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:53.009054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb76dc-b6b2-4ae7-b6d4-189b3dc17f3d None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:53.017074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08c847-386c-4d98-942a-e801f9be531c None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:53.034287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:53.042659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a877852-5ef2-44b5-8a8a-018f8efec188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:53.043263 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a877852-5ef2-44b5-8a8a-018f8efec188 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:53.043535 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a877852-5ef2-44b5-8a8a-018f8efec188 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:53.043833 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a877852-5ef2-44b5-8a8a-018f8efec188 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:53.044480 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a877852-5ef2-44b5-8a8a-018f8efec188 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:53.044779 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a877852-5ef2-44b5-8a8a-018f8efec188 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:53.046142 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a877852-5ef2-44b5-8a8a-018f8efec188 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:53.050167 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9d1d5-deba-4eb8-9165-4b3b1fafd122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:53.050800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9d1d5-deba-4eb8-9165-4b3b1fafd122 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:53.051087 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9d1d5-deba-4eb8-9165-4b3b1fafd122 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:53.051352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9d1d5-deba-4eb8-9165-4b3b1fafd122 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:53.052469 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9d1d5-deba-4eb8-9165-4b3b1fafd122 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:53.053947 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9d1d5-deba-4eb8-9165-4b3b1fafd122 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:53.054039 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 80/151] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:07:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:07:53.054729 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a877852-5ef2-44b5-8a8a-018f8efec188 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:53.058206 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 72/152] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:07:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:07:53.059415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9d1d5-deba-4eb8-9165-4b3b1fafd122 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:53.154311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a877852-5ef2-44b5-8a8a-018f8efec188 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:53.155794 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9d1d5-deba-4eb8-9165-4b3b1fafd122 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:53.186093 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9473a-45d3-440d-9558-510537ca9c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:53.186713 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9473a-45d3-440d-9558-510537ca9c38 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:53.186988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9473a-45d3-440d-9558-510537ca9c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:53.187262 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9473a-45d3-440d-9558-510537ca9c38 None admin] PATH_INFO: `/v3/users/0cf2caec03f0497785352a5b4df31f31` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:53.188659 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741323e-4b61-494b-874e-72558d6d2a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:53.189277 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741323e-4b61-494b-874e-72558d6d2a7f None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:53.189553 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741323e-4b61-494b-874e-72558d6d2a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:53.189828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741323e-4b61-494b-874e-72558d6d2a7f None admin] PATH_INFO: `/v3/users/0cf2caec03f0497785352a5b4df31f31` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:53.200976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:53.202032 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e9473a-45d3-440d-9558-510537ca9c38 None admin] RBAC: Authorizing `identity:get_user(user_id=0cf2caec03f0497785352a5b4df31f31)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:53.203750 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e9473a-45d3-440d-9558-510537ca9c38 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:53.203961 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741323e-4b61-494b-874e-72558d6d2a7f None admin] RBAC: Authorizing `identity:get_user(user_id=0cf2caec03f0497785352a5b4df31f31)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:53.205358 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 81/153] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:53 2022] GET /identity/v3/users/0cf2caec03f0497785352a5b4df31f31 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:53.210229 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741323e-4b61-494b-874e-72558d6d2a7f None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:53.210318 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9473a-45d3-440d-9558-510537ca9c38 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:53.210420 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 73/154] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:53 2022] GET /identity/v3/users/0cf2caec03f0497785352a5b4df31f31 => generated 283 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:07:53.212560 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741323e-4b61-494b-874e-72558d6d2a7f None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:53.251667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:53.304028 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9473a-45d3-440d-9558-510537ca9c38 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:53.307233 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741323e-4b61-494b-874e-72558d6d2a7f None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:53.337513 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5daf0f-5ded-4089-9056-2e4db70b03ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:53.338052 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5daf0f-5ded-4089-9056-2e4db70b03ad None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:53.338335 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5daf0f-5ded-4089-9056-2e4db70b03ad None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:53.338607 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5daf0f-5ded-4089-9056-2e4db70b03ad None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:53.339256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5daf0f-5ded-4089-9056-2e4db70b03ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:53.340057 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e252b6-e7cf-4e6b-a0a0-672d0aac4aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:53.340684 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e252b6-e7cf-4e6b-a0a0-672d0aac4aae None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:53.340780 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5daf0f-5ded-4089-9056-2e4db70b03ad None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:53.340966 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e252b6-e7cf-4e6b-a0a0-672d0aac4aae None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:53.341242 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e252b6-e7cf-4e6b-a0a0-672d0aac4aae None admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:53.342364 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e252b6-e7cf-4e6b-a0a0-672d0aac4aae None admin] RBAC: Authorizing `identity:get_project(project_id=45fcdbb9daa941a89354e8eb723a8fcb)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:53.343880 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e252b6-e7cf-4e6b-a0a0-672d0aac4aae None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:53.345548 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 74/155] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:07:53 2022] GET /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb => generated 302 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:07:53.347421 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e252b6-e7cf-4e6b-a0a0-672d0aac4aae None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:53.368602 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 82/156] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:07:53 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:07:53.369413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5daf0f-5ded-4089-9056-2e4db70b03ad None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:53.423613 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:53.460281 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e252b6-e7cf-4e6b-a0a0-672d0aac4aae None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:53.485980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5daf0f-5ded-4089-9056-2e4db70b03ad None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:53.488188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:53.496821 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93242e2b-3428-4cb3-8836-e0d172ddf9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:53.497644 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93242e2b-3428-4cb3-8836-e0d172ddf9cc None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:53.497939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93242e2b-3428-4cb3-8836-e0d172ddf9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:53.498204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93242e2b-3428-4cb3-8836-e0d172ddf9cc None admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:53.499480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93242e2b-3428-4cb3-8836-e0d172ddf9cc None admin] RBAC: Authorizing `identity:get_project(project_id=45fcdbb9daa941a89354e8eb723a8fcb)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:53.500993 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93242e2b-3428-4cb3-8836-e0d172ddf9cc None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:53.507659 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 75/157] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:07:53 2022] GET /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:53.508791 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93242e2b-3428-4cb3-8836-e0d172ddf9cc None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:53.540705 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96528d79-d086-4218-87b8-643206fb64a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:53.541535 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96528d79-d086-4218-87b8-643206fb64a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:53.541816 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96528d79-d086-4218-87b8-643206fb64a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:53.542083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96528d79-d086-4218-87b8-643206fb64a7 None admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb/users/0cf2caec03f0497785352a5b4df31f31/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:53.544114 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96528d79-d086-4218-87b8-643206fb64a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=45fcdbb9daa941a89354e8eb723a8fcb, user_id=0cf2caec03f0497785352a5b4df31f31, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:53.545569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96528d79-d086-4218-87b8-643206fb64a7 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:53.560366 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 83/158] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:07:53 2022] PUT /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb/users/0cf2caec03f0497785352a5b4df31f31/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:07:53.631673 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93242e2b-3428-4cb3-8836-e0d172ddf9cc None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:53.672603 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecf8b3-0e41-419e-8183-41dc4604f7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:53.673320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecf8b3-0e41-419e-8183-41dc4604f7aa None admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:53.673566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecf8b3-0e41-419e-8183-41dc4604f7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:53.673843 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecf8b3-0e41-419e-8183-41dc4604f7aa None admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb/users/0cf2caec03f0497785352a5b4df31f31/roles/01dbc90c2ce041afa161e621bd524341` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:53.676109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ecf8b3-0e41-419e-8183-41dc4604f7aa None admin] RBAC: Authorizing `identity:create_grant(project_id=45fcdbb9daa941a89354e8eb723a8fcb, user_id=0cf2caec03f0497785352a5b4df31f31, role_id=01dbc90c2ce041afa161e621bd524341)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:53.677608 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ecf8b3-0e41-419e-8183-41dc4604f7aa None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:53.681766 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:53.695331 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 76/159] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:07:53 2022] PUT /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb/users/0cf2caec03f0497785352a5b4df31f31/roles/01dbc90c2ce041afa161e621bd524341 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:07:55.701641 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f4695-60dd-4bcf-90dc-0f4d78fbeb0b None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:55.702257 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f4695-60dd-4bcf-90dc-0f4d78fbeb0b None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:55.702257 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f4695-60dd-4bcf-90dc-0f4d78fbeb0b None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:55.703764 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 84/160] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:55.709501 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cf7e2-79db-4f6d-8916-54a20d7801fb None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:55.709910 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cf7e2-79db-4f6d-8916-54a20d7801fb None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:55.710081 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cf7e2-79db-4f6d-8916-54a20d7801fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:55.728437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-704cf7e2-79db-4f6d-8916-54a20d7801fb None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:55.729622 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-704cf7e2-79db-4f6d-8916-54a20d7801fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:55.773341 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207895b-05cc-4a54-9b74-be379a722385 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:55.773561 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207895b-05cc-4a54-9b74-be379a722385 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:55.773831 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207895b-05cc-4a54-9b74-be379a722385 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:55.775718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 85/161] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:07:55.777307 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa213bc-eb55-4b84-8511-37014e67edff None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:55.777875 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa213bc-eb55-4b84-8511-37014e67edff None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:55.778378 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa213bc-eb55-4b84-8511-37014e67edff None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:55.780253 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 86/162] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:55.781763 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f184f41-7248-477f-9445-6eae3caf44ba None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:55.782294 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f184f41-7248-477f-9445-6eae3caf44ba None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:55.782759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f184f41-7248-477f-9445-6eae3caf44ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:55.805782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-1f184f41-7248-477f-9445-6eae3caf44ba None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:55.806988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1f184f41-7248-477f-9445-6eae3caf44ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:55.808339 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 77/163] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:55.812846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211782b3-7863-40b4-b9cb-12162012cd0b None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:55.813341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211782b3-7863-40b4-b9cb-12162012cd0b None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:55.813818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211782b3-7863-40b4-b9cb-12162012cd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:55.838987 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-211782b3-7863-40b4-b9cb-12162012cd0b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:55.840963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-211782b3-7863-40b4-b9cb-12162012cd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:55.878155 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 87/164] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:07:55.881453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad10fe-163d-491a-b4c0-52c394c22ccb None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:55.881852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad10fe-163d-491a-b4c0-52c394c22ccb None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:55.882212 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad10fe-163d-491a-b4c0-52c394c22ccb None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:55.883847 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 88/165] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:55.888492 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753c697-b13d-4f77-a45e-c2f530119186 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:55.888879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753c697-b13d-4f77-a45e-c2f530119186 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:55.889303 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753c697-b13d-4f77-a45e-c2f530119186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:55.908330 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-1753c697-b13d-4f77-a45e-c2f530119186 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:55.909886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1753c697-b13d-4f77-a45e-c2f530119186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:55.922080 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 78/166] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:55.978081 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 89/167] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:07:56.080709 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211782b3-7863-40b4-b9cb-12162012cd0b None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:56.136335 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:56.155533 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753c697-b13d-4f77-a45e-c2f530119186 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:56.183971 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:56.191224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211782b3-7863-40b4-b9cb-12162012cd0b None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:56.227035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2357c50-d0d3-4d81-ae89-295359e3e888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:56.227842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2357c50-d0d3-4d81-ae89-295359e3e888 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:56.228195 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2357c50-d0d3-4d81-ae89-295359e3e888 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:56.228405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2357c50-d0d3-4d81-ae89-295359e3e888 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:56.238698 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2357c50-d0d3-4d81-ae89-295359e3e888 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:56.244033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-b2357c50-d0d3-4d81-ae89-295359e3e888 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:07:56.244033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:56.244033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:56.244033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:56.244033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:56.244033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:56.244033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:56.244033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:56.244033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:56.244033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:56.244033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:56.244033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:56.244033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:56.244033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:56.244033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:56.244033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:56.245190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:56.245190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:56.245190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:56.245190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:56.245190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:56.245190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:56.245190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:56.245190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:56.245190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:56.245190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:56.245190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:56.245190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:56.245190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:56.245190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:56.245190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:56.245190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:56.246027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:56.246027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:56.246027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:56.246027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:56.246027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:56.246027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:56.246027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:56.246027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:56.246027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:56.246027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:56.246027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:07:56.246027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:07:56.246027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 79/168] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:07:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:07:56.246027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2357c50-d0d3-4d81-ae89-295359e3e888 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:56.263713 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753c697-b13d-4f77-a45e-c2f530119186 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:56.298189 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30390625-74e6-49df-b49a-e268f53f2cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:56.298815 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30390625-74e6-49df-b49a-e268f53f2cdb None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:56.299089 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30390625-74e6-49df-b49a-e268f53f2cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:56.299356 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30390625-74e6-49df-b49a-e268f53f2cdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:56.309011 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30390625-74e6-49df-b49a-e268f53f2cdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:56.313868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-30390625-74e6-49df-b49a-e268f53f2cdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:07:56.313868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:56.313868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:56.313868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:56.313868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:56.313868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:56.313868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:56.313868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:56.313868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:56.313868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:56.313868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:56.313868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:56.313868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:56.313868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:56.313868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:56.313868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:56.314735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:56.314735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:56.314735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:56.314735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:56.314735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:56.314735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:56.314735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:56.314735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:56.314735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:56.314735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:56.314735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:56.314735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:56.314735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:56.314735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:56.314735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:56.314735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:56.316234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:56.316234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:56.316234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:56.316234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:56.316234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:56.316234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:56.316234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:56.316234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:56.316234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:56.316234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:56.316234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:07:56.316234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:07:56.316234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 90/169] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:07:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:07:56.316234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30390625-74e6-49df-b49a-e268f53f2cdb None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:56.346740 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2357c50-d0d3-4d81-ae89-295359e3e888 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:56.362699 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:56.380977 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb0f0f-5ca7-4f2b-8bdc-cbc2a6772c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:56.381602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb0f0f-5ca7-4f2b-8bdc-cbc2a6772c13 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:56.381878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb0f0f-5ca7-4f2b-8bdc-cbc2a6772c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:56.382143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb0f0f-5ca7-4f2b-8bdc-cbc2a6772c13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:56.383223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb0f0f-5ca7-4f2b-8bdc-cbc2a6772c13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:56.384700 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb0f0f-5ca7-4f2b-8bdc-cbc2a6772c13 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:56.386244 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 80/170] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:07:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:56.391083 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb0f0f-5ca7-4f2b-8bdc-cbc2a6772c13 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:56.399075 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:56.416870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30390625-74e6-49df-b49a-e268f53f2cdb None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:56.455419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f9c53-c76a-4c10-94ab-7a71c52f2e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:56.456083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f9c53-c76a-4c10-94ab-7a71c52f2e29 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:56.456361 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f9c53-c76a-4c10-94ab-7a71c52f2e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:56.456634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f9c53-c76a-4c10-94ab-7a71c52f2e29 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:56.466930 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f9c53-c76a-4c10-94ab-7a71c52f2e29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:56.471771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-c93f9c53-c76a-4c10-94ab-7a71c52f2e29 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:07:56.471771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:56.471771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:56.471771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:56.471771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:56.471771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:56.471771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:56.471771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:56.471771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:56.471771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:56.471771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:56.471771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:56.471771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:56.471771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:56.471771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:56.471771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:56.472782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:56.472782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:56.472782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:56.472782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:56.472782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:56.472782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:56.472782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:56.472782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:56.472782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:56.472782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:56.472782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:56.472782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:56.472782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:56.472782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:56.472782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:56.472782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:56.473571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:56.473571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:56.473571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:56.473571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:56.473571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:56.473571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:56.473571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:56.473571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:56.473571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:56.473571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:56.473571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:07:56.473571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:07:56.473571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 91/171] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Oct 21 21:07:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:07:56.474223 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f9c53-c76a-4c10-94ab-7a71c52f2e29 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:56.486062 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb0f0f-5ca7-4f2b-8bdc-cbc2a6772c13 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:56.519080 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367f567-e289-4f30-8017-9c0ed11da1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:56.519719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367f567-e289-4f30-8017-9c0ed11da1bb None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:56.519998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367f567-e289-4f30-8017-9c0ed11da1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:56.520266 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367f567-e289-4f30-8017-9c0ed11da1bb None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:56.520929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367f567-e289-4f30-8017-9c0ed11da1bb None admin] RBAC: Adding query filter params (name=member) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:56.521730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367f567-e289-4f30-8017-9c0ed11da1bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:56.523121 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367f567-e289-4f30-8017-9c0ed11da1bb None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:56.531167 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 81/172] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:07:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:56.531892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367f567-e289-4f30-8017-9c0ed11da1bb None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:56.573786 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f9c53-c76a-4c10-94ab-7a71c52f2e29 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:56.576147 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:56.608890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e223a22-52f8-4eb4-b74a-04dfe6774acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:56.609494 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e223a22-52f8-4eb4-b74a-04dfe6774acc None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:56.609769 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e223a22-52f8-4eb4-b74a-04dfe6774acc None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:56.610033 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e223a22-52f8-4eb4-b74a-04dfe6774acc None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:56.610773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e223a22-52f8-4eb4-b74a-04dfe6774acc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:56.611069 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e223a22-52f8-4eb4-b74a-04dfe6774acc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:56.612490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e223a22-52f8-4eb4-b74a-04dfe6774acc None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:56.620433 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 92/173] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:07:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:56.624550 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e223a22-52f8-4eb4-b74a-04dfe6774acc None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:56.628277 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367f567-e289-4f30-8017-9c0ed11da1bb None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:56.628420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:56.662602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11d90a-71e7-4b12-b7c2-d40e0d804c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:56.663214 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11d90a-71e7-4b12-b7c2-d40e0d804c4b None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:56.663521 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11d90a-71e7-4b12-b7c2-d40e0d804c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:56.663825 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11d90a-71e7-4b12-b7c2-d40e0d804c4b None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:56.664473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c11d90a-71e7-4b12-b7c2-d40e0d804c4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:56.666221 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c11d90a-71e7-4b12-b7c2-d40e0d804c4b None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:56.686150 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 82/174] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:07:56 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:07:56.686150 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11d90a-71e7-4b12-b7c2-d40e0d804c4b None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:56.727008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e223a22-52f8-4eb4-b74a-04dfe6774acc None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:56.764890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac2b6a-decf-4b6c-8f29-c2d63f945bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:56.765696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac2b6a-decf-4b6c-8f29-c2d63f945bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:56.765977 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac2b6a-decf-4b6c-8f29-c2d63f945bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:56.766249 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac2b6a-decf-4b6c-8f29-c2d63f945bf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:56.766934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac2b6a-decf-4b6c-8f29-c2d63f945bf5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:56.767241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac2b6a-decf-4b6c-8f29-c2d63f945bf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:56.768779 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac2b6a-decf-4b6c-8f29-c2d63f945bf5 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:56.777665 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 93/175] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:07:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:56.778627 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac2b6a-decf-4b6c-8f29-c2d63f945bf5 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:56.813924 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11d90a-71e7-4b12-b7c2-d40e0d804c4b None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:56.815863 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:56.853142 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd1394-757f-4f02-9ca3-adb7a947c8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:56.853786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd1394-757f-4f02-9ca3-adb7a947c8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:56.854061 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd1394-757f-4f02-9ca3-adb7a947c8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:56.854330 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd1394-757f-4f02-9ca3-adb7a947c8e6 None admin] PATH_INFO: `/v3/users/4f045c98ab1448b3ba42f5ca6544ed7a` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:56.855641 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:56.870161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd1394-757f-4f02-9ca3-adb7a947c8e6 None admin] RBAC: Authorizing `identity:get_user(user_id=4f045c98ab1448b3ba42f5ca6544ed7a)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:56.871797 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd1394-757f-4f02-9ca3-adb7a947c8e6 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:56.873758 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 83/176] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:56 2022] GET /identity/v3/users/4f045c98ab1448b3ba42f5ca6544ed7a => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:56.874416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd1394-757f-4f02-9ca3-adb7a947c8e6 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:56.897153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac2b6a-decf-4b6c-8f29-c2d63f945bf5 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:56.937064 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c779df1-cd60-40e9-baa9-7473e9b7ee59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:56.937708 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c779df1-cd60-40e9-baa9-7473e9b7ee59 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:56.937990 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c779df1-cd60-40e9-baa9-7473e9b7ee59 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:56.938265 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c779df1-cd60-40e9-baa9-7473e9b7ee59 None admin] PATH_INFO: `/v3/users/0cf2caec03f0497785352a5b4df31f31` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:56.939404 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c779df1-cd60-40e9-baa9-7473e9b7ee59 None admin] RBAC: Authorizing `identity:get_user(user_id=0cf2caec03f0497785352a5b4df31f31)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:56.941079 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c779df1-cd60-40e9-baa9-7473e9b7ee59 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:56.942610 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 94/177] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:56 2022] GET /identity/v3/users/0cf2caec03f0497785352a5b4df31f31 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:56.943333 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c779df1-cd60-40e9-baa9-7473e9b7ee59 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:56.986705 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd1394-757f-4f02-9ca3-adb7a947c8e6 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:57.025612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7bca0-ece6-4029-9796-fef4ab463ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:57.026276 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7bca0-ece6-4029-9796-fef4ab463ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:57.026645 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7bca0-ece6-4029-9796-fef4ab463ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:57.026988 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7bca0-ece6-4029-9796-fef4ab463ab4 None admin] PATH_INFO: `/v3/users/0cf2caec03f0497785352a5b4df31f31` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:57.028246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7bca0-ece6-4029-9796-fef4ab463ab4 None admin] RBAC: Authorizing `identity:get_user(user_id=0cf2caec03f0497785352a5b4df31f31)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:57.029767 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7bca0-ece6-4029-9796-fef4ab463ab4 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:57.031585 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 84/178] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:56 2022] GET /identity/v3/users/0cf2caec03f0497785352a5b4df31f31 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:57.032177 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7bca0-ece6-4029-9796-fef4ab463ab4 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:57.060679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c779df1-cd60-40e9-baa9-7473e9b7ee59 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:57.066987 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:57.108488 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f476be-68e9-4757-8bf4-4e73ac88e86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:57.109737 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f476be-68e9-4757-8bf4-4e73ac88e86d None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:57.110029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f476be-68e9-4757-8bf4-4e73ac88e86d None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:57.110304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f476be-68e9-4757-8bf4-4e73ac88e86d None admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:57.111754 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f476be-68e9-4757-8bf4-4e73ac88e86d None admin] RBAC: Authorizing `identity:get_project(project_id=45fcdbb9daa941a89354e8eb723a8fcb)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:57.113474 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f476be-68e9-4757-8bf4-4e73ac88e86d None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:57.116433 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 95/179] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:07:56 2022] GET /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:57.121296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f476be-68e9-4757-8bf4-4e73ac88e86d None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:57.121296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:57.156370 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7bca0-ece6-4029-9796-fef4ab463ab4 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:57.193768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c3852-8d3e-442e-85f6-345b5f7d3440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:57.194673 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c3852-8d3e-442e-85f6-345b5f7d3440 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:57.195121 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c3852-8d3e-442e-85f6-345b5f7d3440 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:57.195555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c3852-8d3e-442e-85f6-345b5f7d3440 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:57.243077 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f476be-68e9-4757-8bf4-4e73ac88e86d None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:57.278998 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59bd7a-639b-4568-9255-f746a4ff051c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:57.279967 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59bd7a-639b-4568-9255-f746a4ff051c None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:57.280470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59bd7a-639b-4568-9255-f746a4ff051c None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:57.280907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59bd7a-639b-4568-9255-f746a4ff051c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:57.281605 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c3852-8d3e-442e-85f6-345b5f7d3440 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:57.283154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c3852-8d3e-442e-85f6-345b5f7d3440 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:57.317366 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:57.362485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59bd7a-639b-4568-9255-f746a4ff051c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:57.364288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59bd7a-639b-4568-9255-f746a4ff051c None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:57.367353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:57.404145 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 85/180] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:07:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:07:57.409868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c3852-8d3e-442e-85f6-345b5f7d3440 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:57.503694 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 96/181] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:07:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:07:57.503694 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59bd7a-639b-4568-9255-f746a4ff051c None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:57.519849 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c3852-8d3e-442e-85f6-345b5f7d3440 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:57.548454 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:57.558762 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44ed63-122f-44f9-a10d-a6ed5b6ed67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:57.559746 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44ed63-122f-44f9-a10d-a6ed5b6ed67a None admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:57.560240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44ed63-122f-44f9-a10d-a6ed5b6ed67a None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:57.560706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44ed63-122f-44f9-a10d-a6ed5b6ed67a None admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb/users/4f045c98ab1448b3ba42f5ca6544ed7a/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:57.562830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44ed63-122f-44f9-a10d-a6ed5b6ed67a None admin] RBAC: Authorizing `identity:create_grant(project_id=45fcdbb9daa941a89354e8eb723a8fcb, user_id=4f045c98ab1448b3ba42f5ca6544ed7a, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:57.564533 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44ed63-122f-44f9-a10d-a6ed5b6ed67a None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:57.581997 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 86/182] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:07:57 2022] PUT /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb/users/4f045c98ab1448b3ba42f5ca6544ed7a/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:07:57.588043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44ed63-122f-44f9-a10d-a6ed5b6ed67a None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:57.659550 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59bd7a-639b-4568-9255-f746a4ff051c None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:57.665662 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:57.745614 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916b8d9-a64c-4abe-ad0d-5f18b6ead74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:57.745614 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916b8d9-a64c-4abe-ad0d-5f18b6ead74b None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:57.745614 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916b8d9-a64c-4abe-ad0d-5f18b6ead74b None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:57.746346 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916b8d9-a64c-4abe-ad0d-5f18b6ead74b None admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:57.748202 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916b8d9-a64c-4abe-ad0d-5f18b6ead74b None admin] RBAC: Authorizing `identity:get_project(project_id=45fcdbb9daa941a89354e8eb723a8fcb)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:57.749674 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916b8d9-a64c-4abe-ad0d-5f18b6ead74b None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:57.751376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 97/183] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:07:57 2022] GET /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:57.757777 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916b8d9-a64c-4abe-ad0d-5f18b6ead74b None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:57.799877 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44ed63-122f-44f9-a10d-a6ed5b6ed67a None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:57.865802 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a76df2-1459-4226-ae52-a6c384299790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:57.866566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a76df2-1459-4226-ae52-a6c384299790 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:57.866832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a76df2-1459-4226-ae52-a6c384299790 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:57.867214 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a76df2-1459-4226-ae52-a6c384299790 None admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:57.869702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a76df2-1459-4226-ae52-a6c384299790 None admin] RBAC: Authorizing `identity:get_project(project_id=45fcdbb9daa941a89354e8eb723a8fcb)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:57.869953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a76df2-1459-4226-ae52-a6c384299790 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:57.872186 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 87/184] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:07:57 2022] GET /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:57.878278 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a76df2-1459-4226-ae52-a6c384299790 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:57.914126 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916b8d9-a64c-4abe-ad0d-5f18b6ead74b None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:57.953051 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:57.968220 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0f4eb-61b8-45ad-9621-a73f52e2a001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:57.969126 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0f4eb-61b8-45ad-9621-a73f52e2a001 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:57.969633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0f4eb-61b8-45ad-9621-a73f52e2a001 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:57.970098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0f4eb-61b8-45ad-9621-a73f52e2a001 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:57.971009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0f4eb-61b8-45ad-9621-a73f52e2a001 None admin] RBAC: Adding query filter params (role.id=c514b065b7604094b899c586f79b0c5b, scope.project.id=45fcdbb9daa941a89354e8eb723a8fcb, user.id=0cf2caec03f0497785352a5b4df31f31) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:57.971517 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0f4eb-61b8-45ad-9621-a73f52e2a001 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:57.973171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0f4eb-61b8-45ad-9621-a73f52e2a001 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:57.993036 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 98/185] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:07:57 2022] GET /identity/v3/role_assignments?user.id=0cf2caec03f0497785352a5b4df31f31&scope.project.id=45fcdbb9daa941a89354e8eb723a8fcb&role.id=c514b065b7604094b899c586f79b0c5b => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:58.000504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a76df2-1459-4226-ae52-a6c384299790 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:58.045803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4269600-4680-4e96-ba4b-f2b5bc37035d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:58.046429 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4269600-4680-4e96-ba4b-f2b5bc37035d None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:58.046706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4269600-4680-4e96-ba4b-f2b5bc37035d None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:58.046978 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4269600-4680-4e96-ba4b-f2b5bc37035d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:58.047736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4269600-4680-4e96-ba4b-f2b5bc37035d None admin] RBAC: Adding query filter params (role.id=01dbc90c2ce041afa161e621bd524341, scope.project.id=45fcdbb9daa941a89354e8eb723a8fcb, user.id=0cf2caec03f0497785352a5b4df31f31) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:58.048047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4269600-4680-4e96-ba4b-f2b5bc37035d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:58.049516 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4269600-4680-4e96-ba4b-f2b5bc37035d None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:58.061254 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 88/186] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:07:57 2022] GET /identity/v3/role_assignments?user.id=0cf2caec03f0497785352a5b4df31f31&scope.project.id=45fcdbb9daa941a89354e8eb723a8fcb&role.id=01dbc90c2ce041afa161e621bd524341 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:59.055186 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298d31f-30fb-4c4a-9a30-24e3f31d899a None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:59.055684 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298d31f-30fb-4c4a-9a30-24e3f31d899a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:59.055684 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298d31f-30fb-4c4a-9a30-24e3f31d899a None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:59.057169 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 99/187] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:59.062272 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395cf0e-64f8-4028-ab90-d55b5aed3400 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:59.062671 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395cf0e-64f8-4028-ab90-d55b5aed3400 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:59.062827 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395cf0e-64f8-4028-ab90-d55b5aed3400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:59.081010 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-b395cf0e-64f8-4028-ab90-d55b5aed3400 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:59.082175 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b395cf0e-64f8-4028-ab90-d55b5aed3400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:59.087247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99da3c6-ecfc-41c8-9baa-551fcb19568a None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:59.087536 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99da3c6-ecfc-41c8-9baa-551fcb19568a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:59.087878 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99da3c6-ecfc-41c8-9baa-551fcb19568a None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:59.089418 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 100/188] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:07:59.094762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa050d4-db11-4d40-ae4e-0eb1f4229cc1 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:59.095376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa050d4-db11-4d40-ae4e-0eb1f4229cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:59.095676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa050d4-db11-4d40-ae4e-0eb1f4229cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:59.119363 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-caa050d4-db11-4d40-ae4e-0eb1f4229cc1 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:07:59.120911 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-caa050d4-db11-4d40-ae4e-0eb1f4229cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:07:59.151069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 89/189] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:07:59.151238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:59.192646 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 101/190] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:07:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:07:59.418722 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395cf0e-64f8-4028-ab90-d55b5aed3400 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:59.464596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa050d4-db11-4d40-ae4e-0eb1f4229cc1 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:59.468586 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:59.523271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395cf0e-64f8-4028-ab90-d55b5aed3400 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:59.558761 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003ccd4-9830-4207-bce3-25b2b42118d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:59.559390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003ccd4-9830-4207-bce3-25b2b42118d3 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:59.559689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003ccd4-9830-4207-bce3-25b2b42118d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:59.559958 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003ccd4-9830-4207-bce3-25b2b42118d3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:59.569861 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa050d4-db11-4d40-ae4e-0eb1f4229cc1 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:59.570067 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003ccd4-9830-4207-bce3-25b2b42118d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:07:59.574869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-a003ccd4-9830-4207-bce3-25b2b42118d3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 21:07:59.574869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:07:59.574869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:07:59.574869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:07:59.574869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:07:59.574869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:07:59.574869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:07:59.574869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:07:59.574869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:07:59.574869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:07:59.574869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:07:59.574869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:07:59.574869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:07:59.574869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:07:59.574869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:07:59.574869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:07:59.575764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:07:59.575764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:07:59.575764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:07:59.575764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:07:59.575764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:07:59.575764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:07:59.575764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:07:59.575764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:07:59.575764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:07:59.575764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:07:59.575764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:07:59.575764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:07:59.575764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:07:59.575764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:07:59.575764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:07:59.575764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:07:59.576896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:07:59.576896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:07:59.576896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:07:59.576896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:07:59.576896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:07:59.576896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:07:59.576896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:07:59.576896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:07:59.576896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:07:59.576896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:07:59.576896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 21:07:59.576896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:07:59.576896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 90/191] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:07:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:07:59.581578 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003ccd4-9830-4207-bce3-25b2b42118d3 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:59.607192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894eab0-2f20-414c-bef9-16d0e7994ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:59.607968 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894eab0-2f20-414c-bef9-16d0e7994ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:59.608251 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894eab0-2f20-414c-bef9-16d0e7994ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:59.608526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894eab0-2f20-414c-bef9-16d0e7994ad2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:59.609618 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894eab0-2f20-414c-bef9-16d0e7994ad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:59.611068 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894eab0-2f20-414c-bef9-16d0e7994ad2 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:59.612699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 102/192] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:07:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:07:59.618044 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894eab0-2f20-414c-bef9-16d0e7994ad2 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:59.649950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:59.687156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003ccd4-9830-4207-bce3-25b2b42118d3 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:59.698334 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:07:59.723944 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22adbfe5-1ea3-43d5-bc72-d141cd220205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:59.725080 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adbfe5-1ea3-43d5-bc72-d141cd220205 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:59.725358 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adbfe5-1ea3-43d5-bc72-d141cd220205 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:59.725629 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adbfe5-1ea3-43d5-bc72-d141cd220205 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:59.726294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22adbfe5-1ea3-43d5-bc72-d141cd220205 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:07:59.726597 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22adbfe5-1ea3-43d5-bc72-d141cd220205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:59.728026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22adbfe5-1ea3-43d5-bc72-d141cd220205 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:59.731712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894eab0-2f20-414c-bef9-16d0e7994ad2 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:59.736793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 91/193] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:07:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:07:59.745071 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22adbfe5-1ea3-43d5-bc72-d141cd220205 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:07:59.772862 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c3aa5-a98d-4198-a3cb-8bd5ec626a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:59.773473 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c3aa5-a98d-4198-a3cb-8bd5ec626a53 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:59.773757 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c3aa5-a98d-4198-a3cb-8bd5ec626a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:59.774037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c3aa5-a98d-4198-a3cb-8bd5ec626a53 None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:59.774685 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c3aa5-a98d-4198-a3cb-8bd5ec626a53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:59.776149 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c3aa5-a98d-4198-a3cb-8bd5ec626a53 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:59.799256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 103/194] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:07:59 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:07:59.858979 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22adbfe5-1ea3-43d5-bc72-d141cd220205 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:07:59.896872 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433ecf4-8a87-415a-b580-60cf60211f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:07:59.897605 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433ecf4-8a87-415a-b580-60cf60211f4b None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:07:59.897888 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433ecf4-8a87-415a-b580-60cf60211f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:07:59.898162 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433ecf4-8a87-415a-b580-60cf60211f4b None admin] PATH_INFO: `/v3/users/562abf899f47402bbf0a573750c02733` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:07:59.916028 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433ecf4-8a87-415a-b580-60cf60211f4b None admin] RBAC: Authorizing `identity:get_user(user_id=562abf899f47402bbf0a573750c02733)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:07:59.917583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433ecf4-8a87-415a-b580-60cf60211f4b None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:07:59.919616 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 92/195] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:07:59 2022] GET /identity/v3/users/562abf899f47402bbf0a573750c02733 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:07:59.925608 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c3aa5-a98d-4198-a3cb-8bd5ec626a53 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:00.022397 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:00.025726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4f1cd-4e38-4ec4-b719-5ae02b16b500 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:00.026017 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4f1cd-4e38-4ec4-b719-5ae02b16b500 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:00.026290 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4f1cd-4e38-4ec4-b719-5ae02b16b500 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:00.027719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 93/196] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:00.032806 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de4ff2-6cd5-4a23-8be2-80b6370e1b42 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:00.033088 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de4ff2-6cd5-4a23-8be2-80b6370e1b42 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:00.033361 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de4ff2-6cd5-4a23-8be2-80b6370e1b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:00.037687 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c3aa5-a98d-4198-a3cb-8bd5ec626a53 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:00.053602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-a8de4ff2-6cd5-4a23-8be2-80b6370e1b42 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:00.054790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-a8de4ff2-6cd5-4a23-8be2-80b6370e1b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:00.082156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f168c-d5a6-47f2-aa10-c3c05989c018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:00.082863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f168c-d5a6-47f2-aa10-c3c05989c018 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:00.083084 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f168c-d5a6-47f2-aa10-c3c05989c018 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:00.084038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f168c-d5a6-47f2-aa10-c3c05989c018 None admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:00.084884 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:00.085272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f168c-d5a6-47f2-aa10-c3c05989c018 None admin] RBAC: Authorizing `identity:get_project(project_id=45fcdbb9daa941a89354e8eb723a8fcb)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:00.086740 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f168c-d5a6-47f2-aa10-c3c05989c018 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:00.088589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 104/197] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:07:59 2022] GET /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:00.094404 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f168c-d5a6-47f2-aa10-c3c05989c018 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:00.147526 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 94/198] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:00.201036 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f168c-d5a6-47f2-aa10-c3c05989c018 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:00.236026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bf4b4-d4cf-4800-b382-c4c3e6198ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:00.236690 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bf4b4-d4cf-4800-b382-c4c3e6198ff7 None admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:00.236969 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bf4b4-d4cf-4800-b382-c4c3e6198ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:00.237240 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bf4b4-d4cf-4800-b382-c4c3e6198ff7 None admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb/users/562abf899f47402bbf0a573750c02733/roles/c6ec89ead9d24571b777018f6a533293` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:00.239134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9bf4b4-d4cf-4800-b382-c4c3e6198ff7 None admin] RBAC: Authorizing `identity:create_grant(project_id=45fcdbb9daa941a89354e8eb723a8fcb, user_id=562abf899f47402bbf0a573750c02733, role_id=c6ec89ead9d24571b777018f6a533293)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:00.240700 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9bf4b4-d4cf-4800-b382-c4c3e6198ff7 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:00.254785 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 105/199] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:08:00 2022] PUT /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb/users/562abf899f47402bbf0a573750c02733/roles/c6ec89ead9d24571b777018f6a533293 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:08:00.429941 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de4ff2-6cd5-4a23-8be2-80b6370e1b42 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:00.533431 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de4ff2-6cd5-4a23-8be2-80b6370e1b42 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:00.569242 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe30d3-f002-49d5-9d17-29d6a77c981c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:00.569875 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe30d3-f002-49d5-9d17-29d6a77c981c None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:00.570206 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe30d3-f002-49d5-9d17-29d6a77c981c None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:00.570482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe30d3-f002-49d5-9d17-29d6a77c981c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:00.580995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe30d3-f002-49d5-9d17-29d6a77c981c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:00.585828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-6afe30d3-f002-49d5-9d17-29d6a77c981c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:08:00.585828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:00.585828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:00.585828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:00.585828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:00.585828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:00.585828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:00.585828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:00.585828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:00.585828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:00.585828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:00.585828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:00.585828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:00.585828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:00.585828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:00.585828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:00.586674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:00.586674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:00.586674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:00.586674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:00.586674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:00.586674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:00.586674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:00.586674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:00.586674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:00.586674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:00.586674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:00.586674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:00.586674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:00.586674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:00.586674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:00.586674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:00.587452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:00.587452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:00.587452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:00.587452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:00.587452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:00.587452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:00.587452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:00.587452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:00.587452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:00.587452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:00.587452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:08:00.587452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:00.587452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 95/200] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:00.593602 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bf4b4-d4cf-4800-b382-c4c3e6198ff7 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:00.612775 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:00.698003 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bf4b4-d4cf-4800-b382-c4c3e6198ff7 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:00.742034 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b63408-7f37-4135-8899-8a77788b36dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:00.742650 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b63408-7f37-4135-8899-8a77788b36dc None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:00.742926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b63408-7f37-4135-8899-8a77788b36dc None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:00.743201 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b63408-7f37-4135-8899-8a77788b36dc None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:00.743995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b63408-7f37-4135-8899-8a77788b36dc None admin] RBAC: Adding query filter params (name=member) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:00.744292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b63408-7f37-4135-8899-8a77788b36dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:00.745755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b63408-7f37-4135-8899-8a77788b36dc None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:00.754468 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 106/201] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:08:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:00.760249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe30d3-f002-49d5-9d17-29d6a77c981c None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:00.792680 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:00.876221 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe30d3-f002-49d5-9d17-29d6a77c981c None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:00.910357 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9ade4-7f77-45ec-8b4d-6a3b11ea6832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:00.910990 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9ade4-7f77-45ec-8b4d-6a3b11ea6832 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:00.911303 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9ade4-7f77-45ec-8b4d-6a3b11ea6832 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:00.911627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9ade4-7f77-45ec-8b4d-6a3b11ea6832 None admin] PATH_INFO: `/v3/users/4f045c98ab1448b3ba42f5ca6544ed7a` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:00.912792 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9ade4-7f77-45ec-8b4d-6a3b11ea6832 None admin] RBAC: Authorizing `identity:get_user(user_id=4f045c98ab1448b3ba42f5ca6544ed7a)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:00.914348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9ade4-7f77-45ec-8b4d-6a3b11ea6832 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:00.916106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 96/202] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:08:00 2022] GET /identity/v3/users/4f045c98ab1448b3ba42f5ca6544ed7a => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:00.921658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b63408-7f37-4135-8899-8a77788b36dc None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:01.003026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:01.025409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b63408-7f37-4135-8899-8a77788b36dc None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:01.059343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562eb7da-e39e-4d97-bcf5-1733eced5103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:01.059987 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562eb7da-e39e-4d97-bcf5-1733eced5103 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:01.060266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562eb7da-e39e-4d97-bcf5-1733eced5103 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:01.060534 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562eb7da-e39e-4d97-bcf5-1733eced5103 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:01.128411 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562eb7da-e39e-4d97-bcf5-1733eced5103 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:01.129882 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562eb7da-e39e-4d97-bcf5-1733eced5103 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:01.207361 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:01.232231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 107/203] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:08:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:08:01.237919 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9ade4-7f77-45ec-8b4d-6a3b11ea6832 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:01.328051 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:01.340422 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9ade4-7f77-45ec-8b4d-6a3b11ea6832 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:01.375761 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d644c7-656f-4c09-9812-ae3ea6bfbfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:01.376390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d644c7-656f-4c09-9812-ae3ea6bfbfaf None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:01.376663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d644c7-656f-4c09-9812-ae3ea6bfbfaf None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:01.376930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d644c7-656f-4c09-9812-ae3ea6bfbfaf None admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:01.378040 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d644c7-656f-4c09-9812-ae3ea6bfbfaf None admin] RBAC: Authorizing `identity:get_project(project_id=45fcdbb9daa941a89354e8eb723a8fcb)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:01.379491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d644c7-656f-4c09-9812-ae3ea6bfbfaf None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:01.381903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 97/204] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:08:01 2022] GET /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:01.387786 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562eb7da-e39e-4d97-bcf5-1733eced5103 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:01.487658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562eb7da-e39e-4d97-bcf5-1733eced5103 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:01.521764 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02557f8b-0f8a-4cf6-b752-36f864b04ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:01.522425 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02557f8b-0f8a-4cf6-b752-36f864b04ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:01.522705 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02557f8b-0f8a-4cf6-b752-36f864b04ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:01.522975 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02557f8b-0f8a-4cf6-b752-36f864b04ad8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:01.523727 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02557f8b-0f8a-4cf6-b752-36f864b04ad8 None admin] RBAC: Adding query filter params (role.id=c73d9ac0c74147e095a9a038d200e2e9, scope.project.id=45fcdbb9daa941a89354e8eb723a8fcb, user.id=4f045c98ab1448b3ba42f5ca6544ed7a) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:01.524037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02557f8b-0f8a-4cf6-b752-36f864b04ad8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:01.525452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02557f8b-0f8a-4cf6-b752-36f864b04ad8 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:01.531085 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 108/205] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:08:01 2022] GET /identity/v3/role_assignments?user.id=4f045c98ab1448b3ba42f5ca6544ed7a&scope.project.id=45fcdbb9daa941a89354e8eb723a8fcb&role.id=c73d9ac0c74147e095a9a038d200e2e9 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:02.162313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f273fc5-e861-4c19-8231-ddaf5dc27a32 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:02.162686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f273fc5-e861-4c19-8231-ddaf5dc27a32 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:02.162861 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f273fc5-e861-4c19-8231-ddaf5dc27a32 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:02.164405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 98/206] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:02.169539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec4df4-243a-4650-9bf1-ef710a3ca786 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:02.169974 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec4df4-243a-4650-9bf1-ef710a3ca786 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:02.170194 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec4df4-243a-4650-9bf1-ef710a3ca786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:02.187904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-03ec4df4-243a-4650-9bf1-ef710a3ca786 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:02.189091 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-03ec4df4-243a-4650-9bf1-ef710a3ca786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:02.223040 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:02.231135 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24706e67-a42e-45ca-9f88-7174724c1b7e None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:02.231417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24706e67-a42e-45ca-9f88-7174724c1b7e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:02.231718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24706e67-a42e-45ca-9f88-7174724c1b7e None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:02.233197 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 99/207] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:08:02.237978 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc1d76-c2ba-49d9-b05a-f27f663b658c None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:02.238534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc1d76-c2ba-49d9-b05a-f27f663b658c None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:02.238534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc1d76-c2ba-49d9-b05a-f27f663b658c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:02.256459 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-5bfc1d76-c2ba-49d9-b05a-f27f663b658c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:02.257623 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-5bfc1d76-c2ba-49d9-b05a-f27f663b658c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:02.262971 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 109/208] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:02.325192 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 100/209] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:08:02.531786 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec4df4-243a-4650-9bf1-ef710a3ca786 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:02.536331 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02c50a-1e80-43b6-bff1-96f9934ea4c3 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:02.536690 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02c50a-1e80-43b6-bff1-96f9934ea4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:02.536963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02c50a-1e80-43b6-bff1-96f9934ea4c3 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:02.538328 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 101/210] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:08:02.543251 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6fccc-0491-41f7-b3e3-eea456ea1f17 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:02.543531 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6fccc-0491-41f7-b3e3-eea456ea1f17 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:02.543836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6fccc-0491-41f7-b3e3-eea456ea1f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:02.562279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-cef6fccc-0491-41f7-b3e3-eea456ea1f17 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:02.563454 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-cef6fccc-0491-41f7-b3e3-eea456ea1f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:02.632687 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 102/211] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:08:02.633411 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6fccc-0491-41f7-b3e3-eea456ea1f17 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:02.639670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec4df4-243a-4650-9bf1-ef710a3ca786 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:02.649763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:02.679850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde635e7-a00e-4934-b257-b9824286d27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:02.680484 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde635e7-a00e-4934-b257-b9824286d27f None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:02.680769 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde635e7-a00e-4934-b257-b9824286d27f None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:02.681027 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde635e7-a00e-4934-b257-b9824286d27f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:02.691190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde635e7-a00e-4934-b257-b9824286d27f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:02.696589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-cde635e7-a00e-4934-b257-b9824286d27f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:08:02.696589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:02.696589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:02.696589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:02.696589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:02.696589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:02.696589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:02.696589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:02.696589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:02.696589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:02.696589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:02.696589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:02.696589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:02.696589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:02.696589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:02.696589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:02.697511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:02.697511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:02.697511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:02.697511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:02.697511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:02.697511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:02.697511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:02.697511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:02.697511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:02.697511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:02.697511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:02.697511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:02.697511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:02.697511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:02.697511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:02.697511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:02.698980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:02.698980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:02.698980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:02.698980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:02.698980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:02.698980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:02.698980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:02.698980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:02.698980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:02.698980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:02.698980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:08:02.698980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:02.698980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 110/212] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:02.703275 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde635e7-a00e-4934-b257-b9824286d27f None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:02.737763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6fccc-0491-41f7-b3e3-eea456ea1f17 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:02.741538 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:02.773641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f2c0a-1a92-45a2-b595-238f56c74b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:02.774277 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f2c0a-1a92-45a2-b595-238f56c74b4e None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:02.774556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f2c0a-1a92-45a2-b595-238f56c74b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:02.774829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f2c0a-1a92-45a2-b595-238f56c74b4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:02.775958 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f2c0a-1a92-45a2-b595-238f56c74b4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:02.777406 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f2c0a-1a92-45a2-b595-238f56c74b4e None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:02.779276 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 103/213] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:02.784361 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f2c0a-1a92-45a2-b595-238f56c74b4e None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:02.818919 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde635e7-a00e-4934-b257-b9824286d27f None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:02.854605 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17a329-2b46-4a72-b3a8-9d656deca84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:02.855206 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17a329-2b46-4a72-b3a8-9d656deca84e None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:02.855497 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17a329-2b46-4a72-b3a8-9d656deca84e None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:02.855784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17a329-2b46-4a72-b3a8-9d656deca84e None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:02.856811 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17a329-2b46-4a72-b3a8-9d656deca84e None admin] RBAC: Adding query filter params (name=member) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:02.856811 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17a329-2b46-4a72-b3a8-9d656deca84e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:02.858103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17a329-2b46-4a72-b3a8-9d656deca84e None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:02.867452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 111/214] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:08:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:02.871975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:02.872796 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17a329-2b46-4a72-b3a8-9d656deca84e None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:02.889340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f2c0a-1a92-45a2-b595-238f56c74b4e None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:02.922909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c9ecd-c0de-4204-89fe-f3743efa3608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:02.923591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c9ecd-c0de-4204-89fe-f3743efa3608 None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:02.923929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c9ecd-c0de-4204-89fe-f3743efa3608 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:02.924248 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c9ecd-c0de-4204-89fe-f3743efa3608 None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:02.924895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c9ecd-c0de-4204-89fe-f3743efa3608 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:02.926334 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c9ecd-c0de-4204-89fe-f3743efa3608 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:02.947085 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 104/215] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:08:02 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:08:02.947925 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c9ecd-c0de-4204-89fe-f3743efa3608 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:02.978236 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:02.986402 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17a329-2b46-4a72-b3a8-9d656deca84e None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:03.022786 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881e008-37c9-4c4f-86a7-416fa6283c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:03.023439 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881e008-37c9-4c4f-86a7-416fa6283c4f None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:03.023750 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881e008-37c9-4c4f-86a7-416fa6283c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:03.024023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881e008-37c9-4c4f-86a7-416fa6283c4f None admin] PATH_INFO: `/v3/users/8f26b44d170a4fe3bd8c79b8782e2aa0` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:03.038851 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881e008-37c9-4c4f-86a7-416fa6283c4f None admin] RBAC: Authorizing `identity:get_user(user_id=8f26b44d170a4fe3bd8c79b8782e2aa0)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:03.040351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881e008-37c9-4c4f-86a7-416fa6283c4f None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:03.042129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 112/216] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:08:02 2022] GET /identity/v3/users/8f26b44d170a4fe3bd8c79b8782e2aa0 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:03.048033 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881e008-37c9-4c4f-86a7-416fa6283c4f None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:03.061981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c9ecd-c0de-4204-89fe-f3743efa3608 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:03.098269 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa54fb0-7d32-49ee-b3a8-503d27aaa931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:03.099172 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa54fb0-7d32-49ee-b3a8-503d27aaa931 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:03.099457 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa54fb0-7d32-49ee-b3a8-503d27aaa931 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:03.099808 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa54fb0-7d32-49ee-b3a8-503d27aaa931 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:03.109272 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:03.113058 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa54fb0-7d32-49ee-b3a8-503d27aaa931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:03.119204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-2fa54fb0-7d32-49ee-b3a8-503d27aaa931 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 21:08:03.119204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:03.119204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:03.119204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:03.119204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:03.119204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:03.119204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:03.119204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:03.119204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:03.119204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:03.119204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:03.119204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:03.119204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:03.119204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:03.119204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:03.119204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:03.120156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:03.120156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:03.120156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:03.120156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:03.120156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:03.120156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:03.120156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:03.120156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:03.120156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:03.120156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:03.120156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:03.120156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:03.120156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:03.120156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:03.120156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:03.120156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:03.121035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:03.121035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:03.121035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:03.121035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:03.121035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:03.121035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:03.121035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:03.121035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:03.121035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:03.121035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:03.121035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 21:08:03.121035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:03.127724 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 105/217] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:03.129768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa54fb0-7d32-49ee-b3a8-503d27aaa931 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:03.167810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881e008-37c9-4c4f-86a7-416fa6283c4f None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:03.208048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6c34d-b316-4464-9660-73edbc3d3e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:03.209220 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6c34d-b316-4464-9660-73edbc3d3e78 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:03.209220 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6c34d-b316-4464-9660-73edbc3d3e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:03.209348 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6c34d-b316-4464-9660-73edbc3d3e78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:03.210014 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6c34d-b316-4464-9660-73edbc3d3e78 None admin] RBAC: Adding query filter params (role.id=c73d9ac0c74147e095a9a038d200e2e9, user.id=8f26b44d170a4fe3bd8c79b8782e2aa0, scope.system=all) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:03.210577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6c34d-b316-4464-9660-73edbc3d3e78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:03.212103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6c34d-b316-4464-9660-73edbc3d3e78 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:03.220776 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 113/218] 10.222.0.13 () {56 vars in 1308 bytes} [Fri Oct 21 21:08:03 2022] GET /identity/v3/role_assignments?user.id=8f26b44d170a4fe3bd8c79b8782e2aa0&scope.system=all&role.id=c73d9ac0c74147e095a9a038d200e2e9 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:03.257575 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa54fb0-7d32-49ee-b3a8-503d27aaa931 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:03.297002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdb932-1a27-4282-aa22-584fcd437e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:03.297553 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdb932-1a27-4282-aa22-584fcd437e25 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:03.297834 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdb932-1a27-4282-aa22-584fcd437e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:03.298110 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdb932-1a27-4282-aa22-584fcd437e25 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:03.298785 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bdb932-1a27-4282-aa22-584fcd437e25 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:03.299087 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bdb932-1a27-4282-aa22-584fcd437e25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:03.300530 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bdb932-1a27-4282-aa22-584fcd437e25 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:03.311615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 106/219] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:08:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:03.316932 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6c34d-b316-4464-9660-73edbc3d3e78 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:03.337160 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:03.433807 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6c34d-b316-4464-9660-73edbc3d3e78 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:03.470283 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11497432-9228-4db5-b61e-ed8f37008103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:03.470917 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11497432-9228-4db5-b61e-ed8f37008103 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:03.471248 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11497432-9228-4db5-b61e-ed8f37008103 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:03.471524 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11497432-9228-4db5-b61e-ed8f37008103 None admin] PATH_INFO: `/v3/users/562abf899f47402bbf0a573750c02733` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:03.472725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11497432-9228-4db5-b61e-ed8f37008103 None admin] RBAC: Authorizing `identity:get_user(user_id=562abf899f47402bbf0a573750c02733)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:03.474662 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11497432-9228-4db5-b61e-ed8f37008103 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:03.476689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 114/220] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:08:03 2022] GET /identity/v3/users/562abf899f47402bbf0a573750c02733 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:03.484543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdb932-1a27-4282-aa22-584fcd437e25 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:03.549864 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:03.591610 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdb932-1a27-4282-aa22-584fcd437e25 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:03.626051 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cffc0f-0cf7-48a3-bfcf-2ec738942a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:03.627056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cffc0f-0cf7-48a3-bfcf-2ec738942a64 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:03.627532 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cffc0f-0cf7-48a3-bfcf-2ec738942a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:03.628026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cffc0f-0cf7-48a3-bfcf-2ec738942a64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:03.697686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cffc0f-0cf7-48a3-bfcf-2ec738942a64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:03.699158 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cffc0f-0cf7-48a3-bfcf-2ec738942a64 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:03.765165 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:03.808401 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 107/221] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:08:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:08:03.813831 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11497432-9228-4db5-b61e-ed8f37008103 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:03.906018 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:03.923376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11497432-9228-4db5-b61e-ed8f37008103 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:03.960133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41788f04-282c-46cb-826d-f23624fe0717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:03.960783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41788f04-282c-46cb-826d-f23624fe0717 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:03.961065 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41788f04-282c-46cb-826d-f23624fe0717 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:03.961338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41788f04-282c-46cb-826d-f23624fe0717 None admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:03.962486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41788f04-282c-46cb-826d-f23624fe0717 None admin] RBAC: Authorizing `identity:get_project(project_id=45fcdbb9daa941a89354e8eb723a8fcb)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:03.964038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41788f04-282c-46cb-826d-f23624fe0717 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:03.966078 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 115/222] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:08:03 2022] GET /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:03.972272 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cffc0f-0cf7-48a3-bfcf-2ec738942a64 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:04.072132 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cffc0f-0cf7-48a3-bfcf-2ec738942a64 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:04.108478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acff7c6-0535-4162-82aa-b2111e694cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:04.109094 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acff7c6-0535-4162-82aa-b2111e694cab None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:04.109372 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acff7c6-0535-4162-82aa-b2111e694cab None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:04.109636 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acff7c6-0535-4162-82aa-b2111e694cab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:04.110352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acff7c6-0535-4162-82aa-b2111e694cab None admin] RBAC: Adding query filter params (role.id=c6ec89ead9d24571b777018f6a533293, scope.project.id=45fcdbb9daa941a89354e8eb723a8fcb, user.id=562abf899f47402bbf0a573750c02733) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:04.110653 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acff7c6-0535-4162-82aa-b2111e694cab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:04.112113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acff7c6-0535-4162-82aa-b2111e694cab None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:04.118159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 108/223] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:08:03 2022] GET /identity/v3/role_assignments?user.id=562abf899f47402bbf0a573750c02733&scope.project.id=45fcdbb9daa941a89354e8eb723a8fcb&role.id=c6ec89ead9d24571b777018f6a533293 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:05.244052 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2385e-5198-46bd-9dec-4956eb5a5792 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:05.244581 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2385e-5198-46bd-9dec-4956eb5a5792 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:05.244581 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2385e-5198-46bd-9dec-4956eb5a5792 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:05.245987 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 116/224] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:05.252168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85728a-b0af-4157-b62e-b61493445439 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:05.252674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85728a-b0af-4157-b62e-b61493445439 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:05.252745 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85728a-b0af-4157-b62e-b61493445439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:05.272535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-8c85728a-b0af-4157-b62e-b61493445439 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:05.273739 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8c85728a-b0af-4157-b62e-b61493445439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:05.295405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:05.346259 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 109/225] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:05.367540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480f8c1-a417-450d-b2f9-7ac98272922f None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:05.367858 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480f8c1-a417-450d-b2f9-7ac98272922f None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:05.368120 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480f8c1-a417-450d-b2f9-7ac98272922f None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:05.369611 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 117/226] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:05.374599 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1ac24-34d2-443f-8a30-9ea806b689cb None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:05.374887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1ac24-34d2-443f-8a30-9ea806b689cb None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:05.375161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1ac24-34d2-443f-8a30-9ea806b689cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:05.393843 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-09c1ac24-34d2-443f-8a30-9ea806b689cb None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:05.395015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-09c1ac24-34d2-443f-8a30-9ea806b689cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:05.448458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d35ce-5f39-466e-b163-287546fa5551 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:05.448737 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d35ce-5f39-466e-b163-287546fa5551 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:05.449011 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d35ce-5f39-466e-b163-287546fa5551 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:05.450472 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 118/227] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:08:05.454980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230f0e9-7602-4556-8b7a-1df69fa4915d None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:05.455285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230f0e9-7602-4556-8b7a-1df69fa4915d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:05.455553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230f0e9-7602-4556-8b7a-1df69fa4915d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:05.462157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 110/228] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:05.473176 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-9230f0e9-7602-4556-8b7a-1df69fa4915d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:05.474368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9230f0e9-7602-4556-8b7a-1df69fa4915d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:05.540212 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 119/229] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:08:05.617699 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1ac24-34d2-443f-8a30-9ea806b689cb None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:05.699989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:05.715716 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1ac24-34d2-443f-8a30-9ea806b689cb None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:05.730038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230f0e9-7602-4556-8b7a-1df69fa4915d None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:05.754252 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63a38c-625c-401c-8c5e-3831c7af50e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:05.754890 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63a38c-625c-401c-8c5e-3831c7af50e6 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:05.755810 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63a38c-625c-401c-8c5e-3831c7af50e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:05.756007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63a38c-625c-401c-8c5e-3831c7af50e6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:05.766068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63a38c-625c-401c-8c5e-3831c7af50e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:05.770975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-2f63a38c-625c-401c-8c5e-3831c7af50e6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 21:08:05.770975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:05.770975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:05.770975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:05.770975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:05.770975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:05.770975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:05.770975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:05.770975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:05.770975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:05.770975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:05.770975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:05.770975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:05.770975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:05.770975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:05.770975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:05.771903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:05.771903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:05.771903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:05.771903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:05.771903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:05.771903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:05.771903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:05.771903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:05.771903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:05.771903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:05.771903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:05.771903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:05.771903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:05.771903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:05.771903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:05.771903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:05.772686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:05.772686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:05.772686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:05.772686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:05.772686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:05.772686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:05.772686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:05.772686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:05.772686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:05.772686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:05.772686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 21:08:05.772686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:05.772686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 111/230] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:05.777673 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63a38c-625c-401c-8c5e-3831c7af50e6 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:05.822633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:05.840431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230f0e9-7602-4556-8b7a-1df69fa4915d None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:05.872934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc34464-03a4-48a2-bd25-5b94d80726cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:05.873559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc34464-03a4-48a2-bd25-5b94d80726cb None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:05.873844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc34464-03a4-48a2-bd25-5b94d80726cb None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:05.874148 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc34464-03a4-48a2-bd25-5b94d80726cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:05.875324 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc34464-03a4-48a2-bd25-5b94d80726cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:05.876811 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc34464-03a4-48a2-bd25-5b94d80726cb None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:05.877456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63a38c-625c-401c-8c5e-3831c7af50e6 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:05.878392 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 120/231] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:05.878392 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc34464-03a4-48a2-bd25-5b94d80726cb None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:05.911979 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465260e-7b80-4e19-82c8-eb9f4829ccb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:05.912579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465260e-7b80-4e19-82c8-eb9f4829ccb6 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:05.912852 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465260e-7b80-4e19-82c8-eb9f4829ccb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:05.913113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465260e-7b80-4e19-82c8-eb9f4829ccb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:05.913757 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465260e-7b80-4e19-82c8-eb9f4829ccb6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:05.914056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465260e-7b80-4e19-82c8-eb9f4829ccb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:05.915417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465260e-7b80-4e19-82c8-eb9f4829ccb6 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:05.923094 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 112/232] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:08:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:05.923870 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465260e-7b80-4e19-82c8-eb9f4829ccb6 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:05.935976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:05.976392 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc34464-03a4-48a2-bd25-5b94d80726cb None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:06.008760 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f328e-5b25-4008-8dcb-ebcf684c158f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:06.009375 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f328e-5b25-4008-8dcb-ebcf684c158f None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:06.009650 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f328e-5b25-4008-8dcb-ebcf684c158f None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:06.009921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f328e-5b25-4008-8dcb-ebcf684c158f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:06.021501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f328e-5b25-4008-8dcb-ebcf684c158f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:06.022491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465260e-7b80-4e19-82c8-eb9f4829ccb6 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:06.026302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-f21f328e-5b25-4008-8dcb-ebcf684c158f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:08:06.026302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:06.026302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:06.026302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:06.026302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:06.026302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:06.026302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:06.026302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:06.026302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:06.026302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:06.026302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:06.026302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:06.026302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:06.026302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:06.026302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:06.026302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:06.027230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:06.027230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:06.027230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:06.027230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:06.027230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:06.027230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:06.027230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:06.027230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:06.027230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:06.027230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:06.027230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:06.027230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:06.027230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:06.027230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:06.027230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:06.027230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:06.028171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:06.028171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:06.028171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:06.028171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:06.028171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:06.028171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:06.028171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:06.028171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:06.028171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:06.028171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:06.028171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:08:06.028171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:06.028171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 121/233] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:06.028171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f328e-5b25-4008-8dcb-ebcf684c158f None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:06.040812 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:06.059542 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d42ac0-c5f9-4515-8957-4869f3e15634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:06.060182 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d42ac0-c5f9-4515-8957-4869f3e15634 None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:06.060462 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d42ac0-c5f9-4515-8957-4869f3e15634 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:06.060730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d42ac0-c5f9-4515-8957-4869f3e15634 None admin] PATH_INFO: `/v3/groups` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:06.061351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d42ac0-c5f9-4515-8957-4869f3e15634 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:06.062790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d42ac0-c5f9-4515-8957-4869f3e15634 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:06.078190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 113/234] 10.222.0.13 () {60 vars in 1134 bytes} [Fri Oct 21 21:08:05 2022] POST /identity/v3/groups => generated 229 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:08:06.078916 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d42ac0-c5f9-4515-8957-4869f3e15634 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:06.136934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f328e-5b25-4008-8dcb-ebcf684c158f None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:06.164435 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:06.177073 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d2c35a-5e43-4630-96bb-40b1d1573d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:06.177893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2c35a-5e43-4630-96bb-40b1d1573d58 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:06.178243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2c35a-5e43-4630-96bb-40b1d1573d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:06.178580 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2c35a-5e43-4630-96bb-40b1d1573d58 None admin] PATH_INFO: `/v3/users/e269df0ceb5b4dfaa9ee5bc846917c8e` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:06.197262 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d2c35a-5e43-4630-96bb-40b1d1573d58 None admin] RBAC: Authorizing `identity:get_user(user_id=e269df0ceb5b4dfaa9ee5bc846917c8e)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:06.199019 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d2c35a-5e43-4630-96bb-40b1d1573d58 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:06.201753 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 122/235] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:08:06 2022] GET /identity/v3/users/e269df0ceb5b4dfaa9ee5bc846917c8e => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:06.202826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d42ac0-c5f9-4515-8957-4869f3e15634 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:06.207841 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d2c35a-5e43-4630-96bb-40b1d1573d58 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:06.246422 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebe66f-cbc4-4d57-96f9-7df53670a059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:06.247109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebe66f-cbc4-4d57-96f9-7df53670a059 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:06.247397 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebe66f-cbc4-4d57-96f9-7df53670a059 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:06.247706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebe66f-cbc4-4d57-96f9-7df53670a059 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:06.248374 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ebe66f-cbc4-4d57-96f9-7df53670a059 None admin] RBAC: Adding query filter params (name=member) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:06.248677 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ebe66f-cbc4-4d57-96f9-7df53670a059 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:06.250075 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ebe66f-cbc4-4d57-96f9-7df53670a059 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:06.259489 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 114/236] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:08:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:06.266488 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebe66f-cbc4-4d57-96f9-7df53670a059 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:06.282798 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:06.319407 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d2c35a-5e43-4630-96bb-40b1d1573d58 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:06.355835 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa281bb2-9280-42b8-8868-4bae678b9167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:06.356364 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa281bb2-9280-42b8-8868-4bae678b9167 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:06.356649 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa281bb2-9280-42b8-8868-4bae678b9167 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:06.356922 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa281bb2-9280-42b8-8868-4bae678b9167 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:06.357648 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa281bb2-9280-42b8-8868-4bae678b9167 None admin] RBAC: Adding query filter params (role.id=c6ec89ead9d24571b777018f6a533293, user.id=e269df0ceb5b4dfaa9ee5bc846917c8e, scope.system=all) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:06.357957 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa281bb2-9280-42b8-8868-4bae678b9167 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:06.359389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa281bb2-9280-42b8-8868-4bae678b9167 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:06.364457 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 123/237] 10.222.0.13 () {56 vars in 1308 bytes} [Fri Oct 21 21:08:06 2022] GET /identity/v3/role_assignments?user.id=e269df0ceb5b4dfaa9ee5bc846917c8e&scope.system=all&role.id=c6ec89ead9d24571b777018f6a533293 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:06.384944 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebe66f-cbc4-4d57-96f9-7df53670a059 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:06.417347 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:06.425773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a087dc50-dcd1-49cc-a7c4-0531bf5cc881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:06.426777 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087dc50-dcd1-49cc-a7c4-0531bf5cc881 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:06.427295 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087dc50-dcd1-49cc-a7c4-0531bf5cc881 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:06.427789 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087dc50-dcd1-49cc-a7c4-0531bf5cc881 None admin] PATH_INFO: `/v3/users/8f26b44d170a4fe3bd8c79b8782e2aa0` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:06.429260 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a087dc50-dcd1-49cc-a7c4-0531bf5cc881 None admin] RBAC: Authorizing `identity:get_user(user_id=8f26b44d170a4fe3bd8c79b8782e2aa0)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:06.430922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a087dc50-dcd1-49cc-a7c4-0531bf5cc881 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:06.432822 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 115/238] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:08:06 2022] GET /identity/v3/users/8f26b44d170a4fe3bd8c79b8782e2aa0 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:06.438599 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa281bb2-9280-42b8-8868-4bae678b9167 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:06.546400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa281bb2-9280-42b8-8868-4bae678b9167 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:06.576544 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:06.582182 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59146132-1a83-4c67-8ec4-9dde388f2e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:06.582836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59146132-1a83-4c67-8ec4-9dde388f2e85 None admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:06.583137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59146132-1a83-4c67-8ec4-9dde388f2e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:06.583959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59146132-1a83-4c67-8ec4-9dde388f2e85 None admin] PATH_INFO: `/v3/system/users/8f26b44d170a4fe3bd8c79b8782e2aa0/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:06.585382 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59146132-1a83-4c67-8ec4-9dde388f2e85 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8f26b44d170a4fe3bd8c79b8782e2aa0, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:06.586793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59146132-1a83-4c67-8ec4-9dde388f2e85 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:06.598091 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 124/239] 10.222.0.13 () {58 vars in 1334 bytes} [Fri Oct 21 21:08:06 2022] PUT /identity/v3/system/users/8f26b44d170a4fe3bd8c79b8782e2aa0/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:08:08.259891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eec30-cd87-4b8f-87df-d4216bd0553f None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:08.260414 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eec30-cd87-4b8f-87df-d4216bd0553f None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:08.260414 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eec30-cd87-4b8f-87df-d4216bd0553f None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:08.261829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 116/240] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:08.267239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19133464-de7c-436b-8d23-37c8c840291e None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:08.267763 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19133464-de7c-436b-8d23-37c8c840291e None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:08.267845 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19133464-de7c-436b-8d23-37c8c840291e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:08.286278 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-19133464-de7c-436b-8d23-37c8c840291e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:08.287463 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-19133464-de7c-436b-8d23-37c8c840291e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:08.354829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 125/241] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:08.528291 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cfa66-4ff3-4945-aa36-80be42e9fb42 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:08.528623 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cfa66-4ff3-4945-aa36-80be42e9fb42 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:08.528794 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cfa66-4ff3-4945-aa36-80be42e9fb42 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:08.530130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 117/242] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:08.535598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bdd34-7788-40aa-8184-921a82d468db None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:08.535907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bdd34-7788-40aa-8184-921a82d468db None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:08.536236 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bdd34-7788-40aa-8184-921a82d468db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:08.554647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-585bdd34-7788-40aa-8184-921a82d468db None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:08.555859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-585bdd34-7788-40aa-8184-921a82d468db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:08.625088 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 126/243] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:08.627348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cfa66-4ff3-4945-aa36-80be42e9fb42 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:08.705890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9400b-8f16-4d57-b90d-de5bcc50349d None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:08.706170 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9400b-8f16-4d57-b90d-de5bcc50349d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:08.706444 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9400b-8f16-4d57-b90d-de5bcc50349d None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:08.707948 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 127/244] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:08:08.713131 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d1b45-f89a-4642-a753-907ffc7ae35a None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:08.713413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d1b45-f89a-4642-a753-907ffc7ae35a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:08.713682 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d1b45-f89a-4642-a753-907ffc7ae35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:08.730661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cfa66-4ff3-4945-aa36-80be42e9fb42 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:08.732046 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-b35d1b45-f89a-4642-a753-907ffc7ae35a None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:08.734229 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-b35d1b45-f89a-4642-a753-907ffc7ae35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:08.764889 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602e384-10e5-4dc8-ac37-07b9d1cd71c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:08.765515 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602e384-10e5-4dc8-ac37-07b9d1cd71c0 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:08.765789 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602e384-10e5-4dc8-ac37-07b9d1cd71c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:08.766047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602e384-10e5-4dc8-ac37-07b9d1cd71c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:08.767155 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602e384-10e5-4dc8-ac37-07b9d1cd71c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:08.768628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602e384-10e5-4dc8-ac37-07b9d1cd71c0 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:08.770433 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 118/245] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:08.776003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602e384-10e5-4dc8-ac37-07b9d1cd71c0 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:08.789179 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:08.808228 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 128/246] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:08:08.844583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:08.882532 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602e384-10e5-4dc8-ac37-07b9d1cd71c0 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:08.903192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d1b45-f89a-4642-a753-907ffc7ae35a None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:08.917071 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f8871-2edf-4031-9b0f-4933c03ba3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:08.917944 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f8871-2edf-4031-9b0f-4933c03ba3bd None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:08.918225 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f8871-2edf-4031-9b0f-4933c03ba3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:08.918492 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f8871-2edf-4031-9b0f-4933c03ba3bd None admin] PATH_INFO: `/v3/groups` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:08.919110 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f8871-2edf-4031-9b0f-4933c03ba3bd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:08.920583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f8871-2edf-4031-9b0f-4933c03ba3bd None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:08.943774 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 119/247] 10.222.0.13 () {60 vars in 1134 bytes} [Fri Oct 21 21:08:08 2022] POST /identity/v3/groups => generated 226 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:08:09.012060 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d1b45-f89a-4642-a753-907ffc7ae35a None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:09.048981 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0dd35-3055-4574-bcd6-c9a49011644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:09.049542 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0dd35-3055-4574-bcd6-c9a49011644c None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:09.049821 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0dd35-3055-4574-bcd6-c9a49011644c None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:09.050090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0dd35-3055-4574-bcd6-c9a49011644c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:09.061505 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0dd35-3055-4574-bcd6-c9a49011644c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:09.066502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-96f0dd35-3055-4574-bcd6-c9a49011644c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 21:08:09.066502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:09.066502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:09.066502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:09.066502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:09.066502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:09.066502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:09.066502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:09.066502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:09.066502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:09.066502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:09.066502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:09.066502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:09.066502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:09.066502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:09.066502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:09.067464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:09.067464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:09.067464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:09.067464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:09.067464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:09.067464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:09.067464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:09.067464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:09.067464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:09.067464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:09.067464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:09.067464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:09.067464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:09.067464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:09.067464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:09.067464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:09.072291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:09.072291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:09.072291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:09.072291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:09.072291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:09.072291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:09.072291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:09.072291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:09.072291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:09.072291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:09.072291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 21:08:09.072291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:09.072291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 129/248] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:08:09.075732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f8871-2edf-4031-9b0f-4933c03ba3bd None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:09.087500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0dd35-3055-4574-bcd6-c9a49011644c None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:09.144180 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:09.196482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f8871-2edf-4031-9b0f-4933c03ba3bd None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:09.205442 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0dd35-3055-4574-bcd6-c9a49011644c None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:09.229185 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:09.245608 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14198a-4ba5-4924-bee6-9d3b420afe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:09.246946 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14198a-4ba5-4924-bee6-9d3b420afe47 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:09.246946 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14198a-4ba5-4924-bee6-9d3b420afe47 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:09.246946 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14198a-4ba5-4924-bee6-9d3b420afe47 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:09.247424 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14198a-4ba5-4924-bee6-9d3b420afe47 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:09.247761 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14198a-4ba5-4924-bee6-9d3b420afe47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:09.249161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14198a-4ba5-4924-bee6-9d3b420afe47 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:09.253901 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d7bf1-5496-48e5-8fe5-cc231791384f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:09.254524 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d7bf1-5496-48e5-8fe5-cc231791384f None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:09.254801 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d7bf1-5496-48e5-8fe5-cc231791384f None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:09.255989 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d7bf1-5496-48e5-8fe5-cc231791384f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:09.261206 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 120/249] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:08:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:09.267072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14198a-4ba5-4924-bee6-9d3b420afe47 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:09.273777 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d7bf1-5496-48e5-8fe5-cc231791384f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:09.279410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-409d7bf1-5496-48e5-8fe5-cc231791384f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:08:09.279410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:09.279410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:09.279410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:09.279410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:09.279410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:09.279410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:09.279410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:09.279410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:09.279410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:09.279410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:09.279410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:09.279410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:09.279410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:09.279410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:09.279410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:09.280351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:09.280351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:09.280351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:09.280351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:09.280351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:09.280351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:09.280351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:09.280351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:09.280351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:09.280351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:09.280351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:09.280351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:09.280351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:09.280351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:09.280351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:09.280351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:09.281134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:09.281134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:09.281134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:09.281134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:09.281134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:09.281134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:09.281134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:09.281134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:09.281134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:09.281134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:09.281134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:08:09.281134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:09.281134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 130/250] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:08:09.286580 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d7bf1-5496-48e5-8fe5-cc231791384f None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:09.397369 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14198a-4ba5-4924-bee6-9d3b420afe47 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:09.405271 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d7bf1-5496-48e5-8fe5-cc231791384f None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:09.416010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:09.443536 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a35856-5420-4d84-a5b8-79418893fd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:09.444713 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a35856-5420-4d84-a5b8-79418893fd35 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:09.445312 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a35856-5420-4d84-a5b8-79418893fd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:09.445809 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a35856-5420-4d84-a5b8-79418893fd35 None admin] PATH_INFO: `/v3/users/e269df0ceb5b4dfaa9ee5bc846917c8e` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:09.446102 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eede4f-e796-430f-bf1d-79a27b8c3b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:09.446724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eede4f-e796-430f-bf1d-79a27b8c3b68 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:09.447045 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eede4f-e796-430f-bf1d-79a27b8c3b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:09.447265 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eede4f-e796-430f-bf1d-79a27b8c3b68 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:09.447966 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eede4f-e796-430f-bf1d-79a27b8c3b68 None admin] RBAC: Adding query filter params (name=member) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:09.448270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eede4f-e796-430f-bf1d-79a27b8c3b68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:09.448403 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a35856-5420-4d84-a5b8-79418893fd35 None admin] RBAC: Authorizing `identity:get_user(user_id=e269df0ceb5b4dfaa9ee5bc846917c8e)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:09.450203 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eede4f-e796-430f-bf1d-79a27b8c3b68 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:09.450870 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a35856-5420-4d84-a5b8-79418893fd35 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:09.452838 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 121/251] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:08:09 2022] GET /identity/v3/users/e269df0ceb5b4dfaa9ee5bc846917c8e => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:09.461634 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a35856-5420-4d84-a5b8-79418893fd35 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:09.467591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 131/252] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:08:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:09.470817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eede4f-e796-430f-bf1d-79a27b8c3b68 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:09.535959 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:09.588301 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a35856-5420-4d84-a5b8-79418893fd35 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:09.614515 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eede4f-e796-430f-bf1d-79a27b8c3b68 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:09.631292 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d158bb-5bc2-44f5-b88d-3e6fb7309236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:09.632002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d158bb-5bc2-44f5-b88d-3e6fb7309236 None admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:09.632283 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d158bb-5bc2-44f5-b88d-3e6fb7309236 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:09.632561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d158bb-5bc2-44f5-b88d-3e6fb7309236 None admin] PATH_INFO: `/v3/system/users/e269df0ceb5b4dfaa9ee5bc846917c8e/roles/c6ec89ead9d24571b777018f6a533293` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:09.633970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d158bb-5bc2-44f5-b88d-3e6fb7309236 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e269df0ceb5b4dfaa9ee5bc846917c8e, role_id=c6ec89ead9d24571b777018f6a533293)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:09.635363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d158bb-5bc2-44f5-b88d-3e6fb7309236 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:09.647667 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 122/253] 10.222.0.13 () {58 vars in 1334 bytes} [Fri Oct 21 21:08:09 2022] PUT /identity/v3/system/users/e269df0ceb5b4dfaa9ee5bc846917c8e/roles/c6ec89ead9d24571b777018f6a533293 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:08:09.658099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2130d-b779-4a09-89c8-7ac243cd084d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:09.658757 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2130d-b779-4a09-89c8-7ac243cd084d None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:09.659040 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2130d-b779-4a09-89c8-7ac243cd084d None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:09.659302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2130d-b779-4a09-89c8-7ac243cd084d None admin] PATH_INFO: `/v3/users/8f26b44d170a4fe3bd8c79b8782e2aa0` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:09.663238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2130d-b779-4a09-89c8-7ac243cd084d None admin] RBAC: Authorizing `identity:get_user(user_id=8f26b44d170a4fe3bd8c79b8782e2aa0)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:09.664825 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2130d-b779-4a09-89c8-7ac243cd084d None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:09.666609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 132/254] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:08:09 2022] GET /identity/v3/users/8f26b44d170a4fe3bd8c79b8782e2aa0 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:09.673560 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d158bb-5bc2-44f5-b88d-3e6fb7309236 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:09.861915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d158bb-5bc2-44f5-b88d-3e6fb7309236 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:09.907858 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:09.942667 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d4aeb-2b9b-40b9-8a45-34800a563e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:09.943717 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d4aeb-2b9b-40b9-8a45-34800a563e06 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:09.944223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d4aeb-2b9b-40b9-8a45-34800a563e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:09.944693 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d4aeb-2b9b-40b9-8a45-34800a563e06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:09.945615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d4aeb-2b9b-40b9-8a45-34800a563e06 None admin] RBAC: Adding query filter params (role.id=c73d9ac0c74147e095a9a038d200e2e9, user.id=8f26b44d170a4fe3bd8c79b8782e2aa0, scope.system=all) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:09.946122 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d4aeb-2b9b-40b9-8a45-34800a563e06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:09.947876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d4aeb-2b9b-40b9-8a45-34800a563e06 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:09.961767 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 123/255] 10.222.0.13 () {56 vars in 1308 bytes} [Fri Oct 21 21:08:09 2022] GET /identity/v3/role_assignments?user.id=8f26b44d170a4fe3bd8c79b8782e2aa0&scope.system=all&role.id=c73d9ac0c74147e095a9a038d200e2e9 => generated 510 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:12.296860 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76dfe5-7bae-4aec-b96b-6a69f640638c None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:12.297861 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76dfe5-7bae-4aec-b96b-6a69f640638c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:12.298349 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76dfe5-7bae-4aec-b96b-6a69f640638c None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:12.300225 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 133/256] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:12.306146 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdd432-7874-4d53-b7de-d637bdf4465d None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:12.307071 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdd432-7874-4d53-b7de-d637bdf4465d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:12.307547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdd432-7874-4d53-b7de-d637bdf4465d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:12.330225 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-2ccdd432-7874-4d53-b7de-d637bdf4465d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:12.331813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2ccdd432-7874-4d53-b7de-d637bdf4465d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:12.416187 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 124/257] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:12.468474 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36666be-9a13-42c1-900c-f90039dc4775 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:12.469119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36666be-9a13-42c1-900c-f90039dc4775 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:12.469593 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36666be-9a13-42c1-900c-f90039dc4775 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:12.471491 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 134/258] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:12.476906 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01217834-e633-4899-9df5-7a58a20b42e3 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:12.477190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01217834-e633-4899-9df5-7a58a20b42e3 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:12.477466 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01217834-e633-4899-9df5-7a58a20b42e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:12.505115 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-01217834-e633-4899-9df5-7a58a20b42e3 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:12.506363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-01217834-e633-4899-9df5-7a58a20b42e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:12.585940 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 125/259] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 21 21:08:12.728959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85120767-16ca-4305-b364-4d2ff465bcd9 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:12.729652 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85120767-16ca-4305-b364-4d2ff465bcd9 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:12.730155 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85120767-16ca-4305-b364-4d2ff465bcd9 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:12.739855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 135/260] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:12 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:12.745591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1f378-7263-4dd9-bf18-1b44287dad87 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:12.746145 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1f378-7263-4dd9-bf18-1b44287dad87 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:12.746613 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1f378-7263-4dd9-bf18-1b44287dad87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:12.746946 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88044a3-6c1e-4052-8502-488bd4551448 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:12.747231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88044a3-6c1e-4052-8502-488bd4551448 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:12.747504 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88044a3-6c1e-4052-8502-488bd4551448 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:12.749199 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 136/261] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:12.754673 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec45c5e-996c-4238-8d88-ea0ef02b58cb None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:12.754963 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec45c5e-996c-4238-8d88-ea0ef02b58cb None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:12.755236 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec45c5e-996c-4238-8d88-ea0ef02b58cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:12.780301 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-6ad1f378-7263-4dd9-bf18-1b44287dad87 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:12.781842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-6ad1f378-7263-4dd9-bf18-1b44287dad87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:12.787873 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-cec45c5e-996c-4238-8d88-ea0ef02b58cb None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:12.789244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-cec45c5e-996c-4238-8d88-ea0ef02b58cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:12.797741 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 126/262] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 21 21:08:12.805815 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 137/263] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 21 21:08:12.824044 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148bfd5-a1c4-4963-a799-40d64fa6043e None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:12.824642 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148bfd5-a1c4-4963-a799-40d64fa6043e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:12.825103 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148bfd5-a1c4-4963-a799-40d64fa6043e None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:12.826627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 127/264] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:12.831894 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfc416-7448-404e-b0b3-0822037b6742 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:12.832176 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfc416-7448-404e-b0b3-0822037b6742 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:12.832445 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfc416-7448-404e-b0b3-0822037b6742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:12.854370 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148bfd5-a1c4-4963-a799-40d64fa6043e None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:12.868544 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-eedfc416-7448-404e-b0b3-0822037b6742 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:12.869762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-eedfc416-7448-404e-b0b3-0822037b6742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:12.881859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 138/265] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 21 21:08:12.928845 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:12.989261 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148bfd5-a1c4-4963-a799-40d64fa6043e None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.026389 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e2d63-a3c6-4e9c-90f3-ec3062716361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.027023 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e2d63-a3c6-4e9c-90f3-ec3062716361 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.027304 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e2d63-a3c6-4e9c-90f3-ec3062716361 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.027593 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e2d63-a3c6-4e9c-90f3-ec3062716361 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.044414 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e2d63-a3c6-4e9c-90f3-ec3062716361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:13.049208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-1f0e2d63-a3c6-4e9c-90f3-ec3062716361 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 21:08:13.049208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:13.049208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:13.049208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:13.049208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:13.049208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:13.049208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:13.049208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:13.049208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:13.049208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:13.049208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:13.049208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:13.049208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:13.049208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:13.049208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:13.049208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:13.050079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:13.050079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:13.050079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:13.050079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:13.050079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:13.050079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:13.050079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:13.050079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:13.050079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:13.050079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:13.050079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:13.050079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:13.050079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:13.050079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:13.050079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:13.050079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:13.050896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:13.050896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:13.050896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:13.050896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:13.050896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:13.050896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:13.050896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:13.050896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:13.050896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:13.050896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:13.050896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 21:08:13.050896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:13.050896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 128/266] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:08:13.056616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfc416-7448-404e-b0b3-0822037b6742 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.095623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:13.184208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e2d63-a3c6-4e9c-90f3-ec3062716361 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.201777 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e2d63-a3c6-4e9c-90f3-ec3062716361 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.206241 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965fbbc-6d1f-4f3f-81b9-741c838a1b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.206878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965fbbc-6d1f-4f3f-81b9-741c838a1b22 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.207158 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965fbbc-6d1f-4f3f-81b9-741c838a1b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.207432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965fbbc-6d1f-4f3f-81b9-741c838a1b22 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.208094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfc416-7448-404e-b0b3-0822037b6742 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.231115 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2965fbbc-6d1f-4f3f-81b9-741c838a1b22 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:13.236939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-2965fbbc-6d1f-4f3f-81b9-741c838a1b22 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:08:13.236939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:13.236939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:13.236939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:13.236939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:13.236939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:13.236939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:13.236939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:13.236939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:13.236939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:13.236939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:13.236939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:13.236939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:13.236939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:13.236939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:13.236939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:13.237854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:13.237854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:13.237854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:13.237854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:13.237854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:13.237854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:13.237854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:13.237854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:13.237854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:13.237854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:13.237854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:13.237854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:13.237854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:13.237854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:13.237854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:13.237854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:13.238631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:13.238631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:13.238631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:13.238631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:13.238631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:13.238631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:13.238631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:13.238631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:13.238631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:13.238631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:13.238631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:08:13.238631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:13.243054 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 129/267] 10.222.0.13 () {56 vars in 1120 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 57 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:08:13.243054 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965fbbc-6d1f-4f3f-81b9-741c838a1b22 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.257795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965fbbc-6d1f-4f3f-81b9-741c838a1b22 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.262321 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e763a-fde5-4a0e-8ac8-54f5e4e5c909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.262958 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e763a-fde5-4a0e-8ac8-54f5e4e5c909 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.263242 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e763a-fde5-4a0e-8ac8-54f5e4e5c909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.263823 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e763a-fde5-4a0e-8ac8-54f5e4e5c909 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.276387 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e763a-fde5-4a0e-8ac8-54f5e4e5c909 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:13.281439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-825e763a-fde5-4a0e-8ac8-54f5e4e5c909 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:08:13.281439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:13.281439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:13.281439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:13.281439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:13.281439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:13.281439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:13.281439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:13.281439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:13.281439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:13.281439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:13.281439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:13.281439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:13.281439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:13.281439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:13.281439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:13.282313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:13.282313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:13.282313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:13.282313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:13.282313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:13.282313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:13.282313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:13.282313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:13.282313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:13.282313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:13.282313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:13.282313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:13.282313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:13.282313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:13.282313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:13.282313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:13.283107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:13.283107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:13.283107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:13.283107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:13.283107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:13.283107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:13.283107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:13.283107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:13.283107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:13.283107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:13.283107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:08:13.283107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:13.283107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 130/268] 10.222.0.13 () {56 vars in 1120 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 42 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:13.284776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e763a-fde5-4a0e-8ac8-54f5e4e5c909 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.288222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b91a6f-d957-4f9c-aae7-73f46ff8a701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.288863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b91a6f-d957-4f9c-aae7-73f46ff8a701 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.289145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b91a6f-d957-4f9c-aae7-73f46ff8a701 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.289548 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b91a6f-d957-4f9c-aae7-73f46ff8a701 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.290224 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b91a6f-d957-4f9c-aae7-73f46ff8a701 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:13.290649 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b91a6f-d957-4f9c-aae7-73f46ff8a701 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.292199 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b91a6f-d957-4f9c-aae7-73f46ff8a701 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.296756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e763a-fde5-4a0e-8ac8-54f5e4e5c909 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.302766 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7782914-86dd-4936-96d2-8b4aac75bc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.303708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7782914-86dd-4936-96d2-8b4aac75bc60 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.304193 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7782914-86dd-4936-96d2-8b4aac75bc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.307612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 139/269] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:13.307612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b91a6f-d957-4f9c-aae7-73f46ff8a701 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.308113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7782914-86dd-4936-96d2-8b4aac75bc60 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.321610 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b91a6f-d957-4f9c-aae7-73f46ff8a701 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.324343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7782914-86dd-4936-96d2-8b4aac75bc60 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:13.326286 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6add36-ff1d-4323-83d8-a766851d3aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.326897 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6add36-ff1d-4323-83d8-a766851d3aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.327232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6add36-ff1d-4323-83d8-a766851d3aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.327600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6add36-ff1d-4323-83d8-a766851d3aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.328387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6add36-ff1d-4323-83d8-a766851d3aa7 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:13.328699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6add36-ff1d-4323-83d8-a766851d3aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.329128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-e7782914-86dd-4936-96d2-8b4aac75bc60 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:08:13.329128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:13.329128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:13.329128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:13.329128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:13.329128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:13.329128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:13.329128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:13.329128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:13.329128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:13.329128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:13.329128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:13.329128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:13.329128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:13.329128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:13.329128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:13.331123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:13.331123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:13.331123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:13.331123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:13.331123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:13.331123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:13.331123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:13.331123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:13.331123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:13.331123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:13.331123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:13.331123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:13.331123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:13.331123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:13.331123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:13.331123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:13.333611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:13.333611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:13.333611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:13.333611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:13.333611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:13.333611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:13.333611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:13.333611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:13.333611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:13.333611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:13.333611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:08:13.333611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:13.335226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: WARNING py.warnings [None req-dd6add36-ff1d-4323-83d8-a766851d3aa7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 21 21:08:13.335226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: warnings.warn(msg) Oct 21 21:08:13.335226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: Oct 21 21:08:13.335226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6add36-ff1d-4323-83d8-a766851d3aa7 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.335760 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 131/270] 10.222.0.13 () {56 vars in 1135 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 50 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:13.335760 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7782914-86dd-4936-96d2-8b4aac75bc60 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.343319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 140/271] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:13.344444 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6add36-ff1d-4323-83d8-a766851d3aa7 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.348027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7782914-86dd-4936-96d2-8b4aac75bc60 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.359885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba263c95-a491-49b9-950c-10d25197c854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.360797 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba263c95-a491-49b9-950c-10d25197c854 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.361319 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba263c95-a491-49b9-950c-10d25197c854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.361784 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba263c95-a491-49b9-950c-10d25197c854 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.362643 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba263c95-a491-49b9-950c-10d25197c854 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:13.363141 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba263c95-a491-49b9-950c-10d25197c854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.365241 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: WARNING py.warnings [None req-ba263c95-a491-49b9-950c-10d25197c854 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 21 21:08:13.365241 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: warnings.warn(msg) Oct 21 21:08:13.365241 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: Oct 21 21:08:13.366511 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba263c95-a491-49b9-950c-10d25197c854 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.376917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 132/272] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:13.382830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba263c95-a491-49b9-950c-10d25197c854 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.393489 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba263c95-a491-49b9-950c-10d25197c854 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.401922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b8e71-4dd6-4ded-a819-f2c8b5ba2ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.403036 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b8e71-4dd6-4ded-a819-f2c8b5ba2ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.403463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b8e71-4dd6-4ded-a819-f2c8b5ba2ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.403922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b8e71-4dd6-4ded-a819-f2c8b5ba2ac9 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.419425 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b8e71-4dd6-4ded-a819-f2c8b5ba2ac9 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:13.425950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-be5b8e71-4dd6-4ded-a819-f2c8b5ba2ac9 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:08:13.425950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:13.425950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:13.425950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:13.425950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:13.425950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:13.425950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:13.425950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:13.425950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:13.425950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:13.425950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:13.425950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:13.425950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:13.425950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:13.425950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:13.425950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:13.426878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:13.426878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:13.426878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:13.426878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:13.426878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:13.426878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:13.426878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:13.426878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:13.426878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:13.426878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:13.426878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:13.426878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:13.426878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:13.426878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:13.426878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:13.426878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:13.427765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:13.427765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:13.427765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:13.427765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:13.427765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:13.427765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:13.427765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:13.427765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:13.427765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:13.427765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:13.427765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:08:13.427765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:13.427765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 133/273] 10.222.0.13 () {56 vars in 1135 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 50 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:13.429097 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b8e71-4dd6-4ded-a819-f2c8b5ba2ac9 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.437286 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:13.443059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b8e71-4dd6-4ded-a819-f2c8b5ba2ac9 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.448033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743442c-cfa2-48ee-91c4-8938be5973ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.448882 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743442c-cfa2-48ee-91c4-8938be5973ac admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.449161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743442c-cfa2-48ee-91c4-8938be5973ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.449430 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743442c-cfa2-48ee-91c4-8938be5973ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.450088 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743442c-cfa2-48ee-91c4-8938be5973ac admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:13.450390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743442c-cfa2-48ee-91c4-8938be5973ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.452282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743442c-cfa2-48ee-91c4-8938be5973ac admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.463705 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 134/274] 10.222.0.13 () {56 vars in 1132 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:13.463705 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743442c-cfa2-48ee-91c4-8938be5973ac admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.469793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6add36-ff1d-4323-83d8-a766851d3aa7 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.478016 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743442c-cfa2-48ee-91c4-8938be5973ac admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.482418 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1fbad-ce6d-405c-ae3c-052f4f479bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.483042 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1fbad-ce6d-405c-ae3c-052f4f479bce admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.483326 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1fbad-ce6d-405c-ae3c-052f4f479bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.483628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1fbad-ce6d-405c-ae3c-052f4f479bce admin admin] PATH_INFO: `/v3/groups/0567312b9f284f2c8cd8a2703553eb5d` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.492045 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1fbad-ce6d-405c-ae3c-052f4f479bce admin admin] RBAC: Authorizing `identity:get_group(group_id=0567312b9f284f2c8cd8a2703553eb5d)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.493074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: WARNING py.warnings [None req-2dc1fbad-ce6d-405c-ae3c-052f4f479bce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 21:08:13.493074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: warnings.warn(msg) Oct 21 21:08:13.493074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: Oct 21 21:08:13.494708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1fbad-ce6d-405c-ae3c-052f4f479bce admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.496583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 135/275] 10.222.0.13 () {56 vars in 1201 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/groups/0567312b9f284f2c8cd8a2703553eb5d => generated 226 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:13.497189 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1fbad-ce6d-405c-ae3c-052f4f479bce admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.506812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1fbad-ce6d-405c-ae3c-052f4f479bce admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.517415 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfea375-9e99-40b0-8e6d-46d23a0f55a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.517585 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa516e92-238f-4d97-a6a3-87a89915e739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.518059 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfea375-9e99-40b0-8e6d-46d23a0f55a3 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.518156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa516e92-238f-4d97-a6a3-87a89915e739 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.518337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfea375-9e99-40b0-8e6d-46d23a0f55a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.518446 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa516e92-238f-4d97-a6a3-87a89915e739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.518614 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfea375-9e99-40b0-8e6d-46d23a0f55a3 None admin] PATH_INFO: `/v3/users/e269df0ceb5b4dfaa9ee5bc846917c8e` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.518703 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa516e92-238f-4d97-a6a3-87a89915e739 admin admin] PATH_INFO: `/v3/groups/0567312b9f284f2c8cd8a2703553eb5d` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.519840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfea375-9e99-40b0-8e6d-46d23a0f55a3 None admin] RBAC: Authorizing `identity:get_user(user_id=e269df0ceb5b4dfaa9ee5bc846917c8e)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.521180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa516e92-238f-4d97-a6a3-87a89915e739 admin admin] RBAC: Authorizing `identity:get_group(group_id=0567312b9f284f2c8cd8a2703553eb5d)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.521818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfea375-9e99-40b0-8e6d-46d23a0f55a3 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.523102 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa516e92-238f-4d97-a6a3-87a89915e739 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.523649 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 141/276] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/users/e269df0ceb5b4dfaa9ee5bc846917c8e => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:13.524252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfea375-9e99-40b0-8e6d-46d23a0f55a3 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.525040 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 136/277] 10.222.0.13 () {56 vars in 1201 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/groups/0567312b9f284f2c8cd8a2703553eb5d => generated 226 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:13.525710 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa516e92-238f-4d97-a6a3-87a89915e739 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.537545 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa516e92-238f-4d97-a6a3-87a89915e739 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.539021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfea375-9e99-40b0-8e6d-46d23a0f55a3 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.541896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf5381-9c9c-4ee6-89c9-e0a266d9bbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.542515 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf5381-9c9c-4ee6-89c9-e0a266d9bbfd admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.542800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf5381-9c9c-4ee6-89c9-e0a266d9bbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.543070 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf5381-9c9c-4ee6-89c9-e0a266d9bbfd admin admin] PATH_INFO: `/v3/groups/0567312b9f284f2c8cd8a2703553eb5d` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.543492 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c12ce-a95a-452a-95c5-0238a73b8f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.544118 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c12ce-a95a-452a-95c5-0238a73b8f4e admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.544210 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf5381-9c9c-4ee6-89c9-e0a266d9bbfd admin admin] RBAC: Authorizing `identity:get_group(group_id=0567312b9f284f2c8cd8a2703553eb5d)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.544427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c12ce-a95a-452a-95c5-0238a73b8f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.544675 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c12ce-a95a-452a-95c5-0238a73b8f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.545342 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c12ce-a95a-452a-95c5-0238a73b8f4e admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:13.545644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c12ce-a95a-452a-95c5-0238a73b8f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.546059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf5381-9c9c-4ee6-89c9-e0a266d9bbfd admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.547254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c12ce-a95a-452a-95c5-0238a73b8f4e admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.547799 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 137/278] 10.222.0.13 () {56 vars in 1201 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/groups/0567312b9f284f2c8cd8a2703553eb5d => generated 226 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:13.548363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf5381-9c9c-4ee6-89c9-e0a266d9bbfd admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.558592 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 142/279] 10.222.0.13 () {56 vars in 1132 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:13.558914 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf5381-9c9c-4ee6-89c9-e0a266d9bbfd admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.560067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c12ce-a95a-452a-95c5-0238a73b8f4e admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.563360 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d55d49-035f-4860-9a13-8c1af8f94a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.564023 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d55d49-035f-4860-9a13-8c1af8f94a8b admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.564320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d55d49-035f-4860-9a13-8c1af8f94a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.565274 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d55d49-035f-4860-9a13-8c1af8f94a8b admin admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.565685 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d55d49-035f-4860-9a13-8c1af8f94a8b admin admin] RBAC: Authorizing `identity:get_project(project_id=dd6450bb3add4920a802300916dc915c)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.567811 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d55d49-035f-4860-9a13-8c1af8f94a8b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.569553 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 138/280] 10.222.0.13 () {56 vars in 1207 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/projects/dd6450bb3add4920a802300916dc915c => generated 298 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:13.570179 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d55d49-035f-4860-9a13-8c1af8f94a8b admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.580554 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d55d49-035f-4860-9a13-8c1af8f94a8b admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.587548 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67732df1-337c-4bda-b58e-a9ac976cdf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.588204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67732df1-337c-4bda-b58e-a9ac976cdf5b admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.588505 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67732df1-337c-4bda-b58e-a9ac976cdf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.588791 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67732df1-337c-4bda-b58e-a9ac976cdf5b admin admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.589937 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67732df1-337c-4bda-b58e-a9ac976cdf5b admin admin] RBAC: Authorizing `identity:get_project(project_id=45fcdbb9daa941a89354e8eb723a8fcb)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.592593 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67732df1-337c-4bda-b58e-a9ac976cdf5b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.594043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 139/281] 10.222.0.13 () {56 vars in 1207 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb => generated 302 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:13.595803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67732df1-337c-4bda-b58e-a9ac976cdf5b admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.605717 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67732df1-337c-4bda-b58e-a9ac976cdf5b admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.610136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f05dc-b450-47dc-b730-03a559c5bf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.610759 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f05dc-b450-47dc-b730-03a559c5bf75 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.611039 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f05dc-b450-47dc-b730-03a559c5bf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.611310 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f05dc-b450-47dc-b730-03a559c5bf75 admin admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.612459 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f05dc-b450-47dc-b730-03a559c5bf75 admin admin] RBAC: Authorizing `identity:get_project(project_id=45fcdbb9daa941a89354e8eb723a8fcb)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.615081 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f05dc-b450-47dc-b730-03a559c5bf75 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.616959 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 140/282] 10.222.0.13 () {56 vars in 1207 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb => generated 302 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:13.617557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f05dc-b450-47dc-b730-03a559c5bf75 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.627287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f05dc-b450-47dc-b730-03a559c5bf75 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.631812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33e21c-a901-484a-b3e3-d49a6aba1538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.632439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33e21c-a901-484a-b3e3-d49a6aba1538 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.632714 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33e21c-a901-484a-b3e3-d49a6aba1538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.632976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33e21c-a901-484a-b3e3-d49a6aba1538 admin admin] PATH_INFO: `/v3/groups/0567312b9f284f2c8cd8a2703553eb5d` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.634069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33e21c-a901-484a-b3e3-d49a6aba1538 admin admin] RBAC: Authorizing `identity:get_group(group_id=0567312b9f284f2c8cd8a2703553eb5d)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.636005 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33e21c-a901-484a-b3e3-d49a6aba1538 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.637602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 141/283] 10.222.0.13 () {56 vars in 1201 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/groups/0567312b9f284f2c8cd8a2703553eb5d => generated 226 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:13.638312 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33e21c-a901-484a-b3e3-d49a6aba1538 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.649079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33e21c-a901-484a-b3e3-d49a6aba1538 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.653510 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87808b79-ecf0-4db1-ba75-3cf508ee26da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.654154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87808b79-ecf0-4db1-ba75-3cf508ee26da admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.654433 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87808b79-ecf0-4db1-ba75-3cf508ee26da admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.654703 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87808b79-ecf0-4db1-ba75-3cf508ee26da admin admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c/groups/0567312b9f284f2c8cd8a2703553eb5d/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.656617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87808b79-ecf0-4db1-ba75-3cf508ee26da admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd6450bb3add4920a802300916dc915c, group_id=0567312b9f284f2c8cd8a2703553eb5d, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.657723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: WARNING py.warnings [None req-87808b79-ecf0-4db1-ba75-3cf508ee26da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 21:08:13.657723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: warnings.warn(msg) Oct 21 21:08:13.657723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: Oct 21 21:08:13.659695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87808b79-ecf0-4db1-ba75-3cf508ee26da admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.664377 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:13.676571 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 142/284] 10.222.0.13 () {58 vars in 1463 bytes} [Fri Oct 21 21:08:13 2022] PUT /identity/v3/projects/dd6450bb3add4920a802300916dc915c/groups/0567312b9f284f2c8cd8a2703553eb5d/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:08:13.679743 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87808b79-ecf0-4db1-ba75-3cf508ee26da admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.687074 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c12ce-a95a-452a-95c5-0238a73b8f4e admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.734597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac8555-3e4f-474c-979c-3867d73c888e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.735370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac8555-3e4f-474c-979c-3867d73c888e None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.735768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac8555-3e4f-474c-979c-3867d73c888e None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.736683 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac8555-3e4f-474c-979c-3867d73c888e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.737004 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac8555-3e4f-474c-979c-3867d73c888e None admin] RBAC: Adding query filter params (role.id=c6ec89ead9d24571b777018f6a533293, user.id=e269df0ceb5b4dfaa9ee5bc846917c8e, scope.system=all) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:13.737965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac8555-3e4f-474c-979c-3867d73c888e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.740363 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac8555-3e4f-474c-979c-3867d73c888e None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.742453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:13.757124 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 143/285] 10.222.0.13 () {56 vars in 1308 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/role_assignments?user.id=e269df0ceb5b4dfaa9ee5bc846917c8e&scope.system=all&role.id=c6ec89ead9d24571b777018f6a533293 => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:13.758088 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac8555-3e4f-474c-979c-3867d73c888e None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.766756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87808b79-ecf0-4db1-ba75-3cf508ee26da admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.772587 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac8555-3e4f-474c-979c-3867d73c888e None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.774392 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec2809-0713-4de1-a8ed-a23906b3e174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.775501 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec2809-0713-4de1-a8ed-a23906b3e174 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.776057 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec2809-0713-4de1-a8ed-a23906b3e174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.776504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec2809-0713-4de1-a8ed-a23906b3e174 admin admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb/groups/0567312b9f284f2c8cd8a2703553eb5d/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.778292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4ad67-0328-4156-bc0a-025387abeac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.779073 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec2809-0713-4de1-a8ed-a23906b3e174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45fcdbb9daa941a89354e8eb723a8fcb, group_id=0567312b9f284f2c8cd8a2703553eb5d, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.779239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4ad67-0328-4156-bc0a-025387abeac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.779686 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4ad67-0328-4156-bc0a-025387abeac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.780060 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4ad67-0328-4156-bc0a-025387abeac7 admin admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb/groups/0567312b9f284f2c8cd8a2703553eb5d/roles/01dbc90c2ce041afa161e621bd524341` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.782503 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e4ad67-0328-4156-bc0a-025387abeac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45fcdbb9daa941a89354e8eb723a8fcb, group_id=0567312b9f284f2c8cd8a2703553eb5d, role_id=01dbc90c2ce041afa161e621bd524341)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.783349 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec2809-0713-4de1-a8ed-a23906b3e174 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.783863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: WARNING py.warnings [None req-72e4ad67-0328-4156-bc0a-025387abeac7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 21:08:13.783863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: warnings.warn(msg) Oct 21 21:08:13.783863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: Oct 21 21:08:13.786268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e4ad67-0328-4156-bc0a-025387abeac7 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.806223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 143/286] 10.222.0.13 () {58 vars in 1463 bytes} [Fri Oct 21 21:08:13 2022] PUT /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb/groups/0567312b9f284f2c8cd8a2703553eb5d/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:08:13.807116 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec2809-0713-4de1-a8ed-a23906b3e174 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.808028 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 144/287] 10.222.0.13 () {58 vars in 1463 bytes} [Fri Oct 21 21:08:13 2022] PUT /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb/groups/0567312b9f284f2c8cd8a2703553eb5d/roles/01dbc90c2ce041afa161e621bd524341 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:08:13.891641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec2809-0713-4de1-a8ed-a23906b3e174 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.961583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a13de-571a-4520-875d-c9fb0dd376a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.962510 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a13de-571a-4520-875d-c9fb0dd376a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.962778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a13de-571a-4520-875d-c9fb0dd376a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.963057 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a13de-571a-4520-875d-c9fb0dd376a3 admin admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.964285 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208a13de-571a-4520-875d-c9fb0dd376a3 admin admin] RBAC: Authorizing `identity:get_project(project_id=dd6450bb3add4920a802300916dc915c)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.966490 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208a13de-571a-4520-875d-c9fb0dd376a3 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:13.968609 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 144/288] 10.222.0.13 () {56 vars in 1207 bytes} [Fri Oct 21 21:08:13 2022] GET /identity/v3/projects/dd6450bb3add4920a802300916dc915c => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:13.974661 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4ad67-0328-4156-bc0a-025387abeac7 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:13.985717 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4ad67-0328-4156-bc0a-025387abeac7 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:13.990672 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ac961-9059-49c7-a924-ad63b5d89032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:13.991369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ac961-9059-49c7-a924-ad63b5d89032 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:13.991685 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ac961-9059-49c7-a924-ad63b5d89032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:13.991958 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ac961-9059-49c7-a924-ad63b5d89032 admin admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c/groups/0567312b9f284f2c8cd8a2703553eb5d/roles/01dbc90c2ce041afa161e621bd524341` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:13.994370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ac961-9059-49c7-a924-ad63b5d89032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd6450bb3add4920a802300916dc915c, group_id=0567312b9f284f2c8cd8a2703553eb5d, role_id=01dbc90c2ce041afa161e621bd524341)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:13.997367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ac961-9059-49c7-a924-ad63b5d89032 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:14.014839 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 145/289] 10.222.0.13 () {58 vars in 1463 bytes} [Fri Oct 21 21:08:13 2022] PUT /identity/v3/projects/dd6450bb3add4920a802300916dc915c/groups/0567312b9f284f2c8cd8a2703553eb5d/roles/01dbc90c2ce041afa161e621bd524341 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:08:14.526942 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66791fa-be8e-480c-ba10-cc4648c266ce None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:14.527452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66791fa-be8e-480c-ba10-cc4648c266ce None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:14.527556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66791fa-be8e-480c-ba10-cc4648c266ce None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:14.529121 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 145/290] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:14.534275 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf1ac7-52b7-486a-9e1a-861d092a0c88 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:14.534694 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf1ac7-52b7-486a-9e1a-861d092a0c88 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:14.534870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf1ac7-52b7-486a-9e1a-861d092a0c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:14.560015 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-f5bf1ac7-52b7-486a-9e1a-861d092a0c88 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:14.561272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-f5bf1ac7-52b7-486a-9e1a-861d092a0c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:14.636994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 146/291] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 21 21:08:15.090952 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66791fa-be8e-480c-ba10-cc4648c266ce None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:15.106694 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66791fa-be8e-480c-ba10-cc4648c266ce None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:15.111007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd679f0a-0345-4b5f-a9ef-33856bc629d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:15.111663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679f0a-0345-4b5f-a9ef-33856bc629d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:15.111940 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679f0a-0345-4b5f-a9ef-33856bc629d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:15.112210 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679f0a-0345-4b5f-a9ef-33856bc629d8 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:15.126307 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd679f0a-0345-4b5f-a9ef-33856bc629d8 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:15.131299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-fd679f0a-0345-4b5f-a9ef-33856bc629d8 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:08:15.131299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:15.131299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:15.131299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:15.131299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:15.131299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:15.131299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:15.131299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:15.131299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:15.131299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:15.131299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:15.131299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:15.131299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:15.131299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:15.131299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:15.131299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:15.132340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:15.132340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:15.132340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:15.132340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:15.132340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:15.132340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:15.132340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:15.132340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:15.132340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:15.132340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:15.132340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:15.132340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:15.132340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:15.132340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:15.132340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:15.132340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:15.133119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:15.133119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:15.133119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:15.133119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:15.133119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:15.133119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:15.133119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:15.133119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:15.133119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:15.133119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:15.133119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:08:15.133119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:15.133119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 146/292] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Oct 21 21:08:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 42 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:15.140577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf1ac7-52b7-486a-9e1a-861d092a0c88 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:15.152598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf1ac7-52b7-486a-9e1a-861d092a0c88 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:15.160149 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3f11e-9dae-4a65-abb0-e978f61b63a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:15.160818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3f11e-9dae-4a65-abb0-e978f61b63a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:15.161106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3f11e-9dae-4a65-abb0-e978f61b63a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:15.161379 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3f11e-9dae-4a65-abb0-e978f61b63a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:15.162040 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3f11e-9dae-4a65-abb0-e978f61b63a7 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:15.162342 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3f11e-9dae-4a65-abb0-e978f61b63a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:15.163968 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3f11e-9dae-4a65-abb0-e978f61b63a7 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:15.172562 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 147/293] 10.222.0.13 () {56 vars in 1120 bytes} [Fri Oct 21 21:08:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:15.178960 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd679f0a-0345-4b5f-a9ef-33856bc629d8 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:15.192605 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd679f0a-0345-4b5f-a9ef-33856bc629d8 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:15.200680 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783582f1-3fd6-483d-b7ab-2092bbed8940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:15.201311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783582f1-3fd6-483d-b7ab-2092bbed8940 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:15.201583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783582f1-3fd6-483d-b7ab-2092bbed8940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:15.201848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783582f1-3fd6-483d-b7ab-2092bbed8940 admin admin] PATH_INFO: `/v3/groups/d7b7c44b80564b448289ccfac290425e` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:15.207018 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783582f1-3fd6-483d-b7ab-2092bbed8940 admin admin] RBAC: Authorizing `identity:get_group(group_id=d7b7c44b80564b448289ccfac290425e)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:15.209479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783582f1-3fd6-483d-b7ab-2092bbed8940 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:15.211359 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 147/294] 10.222.0.13 () {56 vars in 1201 bytes} [Fri Oct 21 21:08:15 2022] GET /identity/v3/groups/d7b7c44b80564b448289ccfac290425e => generated 229 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:15.216785 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3f11e-9dae-4a65-abb0-e978f61b63a7 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:15.232516 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3f11e-9dae-4a65-abb0-e978f61b63a7 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:15.240162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad051553-5ba9-4629-b863-548d8a841b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:15.240800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad051553-5ba9-4629-b863-548d8a841b68 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:15.241077 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad051553-5ba9-4629-b863-548d8a841b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:15.241346 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad051553-5ba9-4629-b863-548d8a841b68 admin admin] PATH_INFO: `/v3/projects/64a13af16ef54c01b2a28b4530062691` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:15.242487 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad051553-5ba9-4629-b863-548d8a841b68 admin admin] RBAC: Authorizing `identity:get_project(project_id=64a13af16ef54c01b2a28b4530062691)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:15.244291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad051553-5ba9-4629-b863-548d8a841b68 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:15.246067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 148/295] 10.222.0.13 () {56 vars in 1207 bytes} [Fri Oct 21 21:08:15 2022] GET /identity/v3/projects/64a13af16ef54c01b2a28b4530062691 => generated 344 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:15.252125 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783582f1-3fd6-483d-b7ab-2092bbed8940 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:15.264789 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783582f1-3fd6-483d-b7ab-2092bbed8940 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:15.269195 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253b445-4337-4339-a85e-fdb487e2c8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:15.269847 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253b445-4337-4339-a85e-fdb487e2c8c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:15.270638 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253b445-4337-4339-a85e-fdb487e2c8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:15.270908 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253b445-4337-4339-a85e-fdb487e2c8c7 admin admin] PATH_INFO: `/v3/projects/64a13af16ef54c01b2a28b4530062691/groups/d7b7c44b80564b448289ccfac290425e/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:15.272787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253b445-4337-4339-a85e-fdb487e2c8c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a13af16ef54c01b2a28b4530062691, group_id=d7b7c44b80564b448289ccfac290425e, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:15.275504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253b445-4337-4339-a85e-fdb487e2c8c7 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:15.297184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 148/296] 10.222.0.13 () {58 vars in 1463 bytes} [Fri Oct 21 21:08:15 2022] PUT /identity/v3/projects/64a13af16ef54c01b2a28b4530062691/groups/d7b7c44b80564b448289ccfac290425e/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:08:16.507755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f880b-9094-4b9d-af7e-aac549f7dcc7 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:16.508264 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f880b-9094-4b9d-af7e-aac549f7dcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:16.508339 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f880b-9094-4b9d-af7e-aac549f7dcc7 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:16.509896 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 149/297] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:16.515787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69d91c-7049-4468-b5ed-dc32e8cf19d6 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:16.516608 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69d91c-7049-4468-b5ed-dc32e8cf19d6 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:16.517082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69d91c-7049-4468-b5ed-dc32e8cf19d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:16.535165 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:16.552869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-ee69d91c-7049-4468-b5ed-dc32e8cf19d6 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:16.554237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ee69d91c-7049-4468-b5ed-dc32e8cf19d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:16.555868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcc41e-be1d-49a4-9156-1c4d906a4777 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:16.556384 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcc41e-be1d-49a4-9156-1c4d906a4777 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:16.556859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcc41e-be1d-49a4-9156-1c4d906a4777 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:16.558743 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 150/298] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:16.568616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483738e-de73-401c-8638-863097f95ce6 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:16.568903 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483738e-de73-401c-8638-863097f95ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:16.569183 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483738e-de73-401c-8638-863097f95ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:16.607793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-d483738e-de73-401c-8638-863097f95ce6 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:16.609194 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d483738e-de73-401c-8638-863097f95ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:16.633069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 149/299] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:08:16.715018 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 151/300] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:16.723133 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290f650-fa0c-4762-8359-4a9664e514d1 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:16.723767 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290f650-fa0c-4762-8359-4a9664e514d1 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:16.724278 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290f650-fa0c-4762-8359-4a9664e514d1 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:16.725827 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 150/301] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:16.731941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e623092-2138-4beb-b997-38a4afc6b37a None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:16.732230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e623092-2138-4beb-b997-38a4afc6b37a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:16.732508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e623092-2138-4beb-b997-38a4afc6b37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:16.753497 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-7e623092-2138-4beb-b997-38a4afc6b37a None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:16.754753 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-7e623092-2138-4beb-b997-38a4afc6b37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:16.791333 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:16.848657 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 152/302] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:16.982574 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290f650-fa0c-4762-8359-4a9664e514d1 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:17.068232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e623092-2138-4beb-b997-38a4afc6b37a None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:17.092926 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290f650-fa0c-4762-8359-4a9664e514d1 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:17.131546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea2cb4-5afb-4719-8189-a7e56766cef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:17.132209 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea2cb4-5afb-4719-8189-a7e56766cef1 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:17.132486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea2cb4-5afb-4719-8189-a7e56766cef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:17.132757 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea2cb4-5afb-4719-8189-a7e56766cef1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:17.134686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:17.146456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea2cb4-5afb-4719-8189-a7e56766cef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:17.151602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-7dea2cb4-5afb-4719-8189-a7e56766cef1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:08:17.151602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:17.151602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:17.151602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:17.151602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:17.151602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:17.151602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:17.151602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:17.151602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:17.151602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:17.151602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:17.151602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:17.151602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:17.151602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:17.151602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:17.151602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:17.152669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:17.152669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:17.152669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:17.152669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:17.152669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:17.152669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:17.152669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:17.152669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:17.152669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:17.152669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:17.152669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:17.152669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:17.152669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:17.152669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:17.152669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:17.152669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:17.153583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:17.153583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:17.153583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:17.153583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:17.153583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:17.153583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:17.153583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:17.153583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:17.153583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:17.153583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:17.153583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:08:17.153583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:17.155479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 151/303] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Oct 21 21:08:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:17.165070 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea2cb4-5afb-4719-8189-a7e56766cef1 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:17.244531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e623092-2138-4beb-b997-38a4afc6b37a None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:17.287081 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c61ec-cb4d-4a7b-93b0-0c01157a415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:17.287903 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c61ec-cb4d-4a7b-93b0-0c01157a415e None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:17.288185 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c61ec-cb4d-4a7b-93b0-0c01157a415e None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:17.288458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c61ec-cb4d-4a7b-93b0-0c01157a415e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:17.299089 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c61ec-cb4d-4a7b-93b0-0c01157a415e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:17.305311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-ec6c61ec-cb4d-4a7b-93b0-0c01157a415e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:08:17.305311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:17.305311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:17.305311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:17.305311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:17.305311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:17.305311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:17.305311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:17.305311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:17.305311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:17.305311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:17.305311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:17.305311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:17.305311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:17.305311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:17.305311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:17.306214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:17.306214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:17.306214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:17.306214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:17.306214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:17.306214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:17.306214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:17.306214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:17.306214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:17.306214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:17.306214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:17.306214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:17.306214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:17.306214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:17.306214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:17.306214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:17.306965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:17.306965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:17.306965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:17.306965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:17.306965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:17.306965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:17.306965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:17.306965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:17.306965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:17.306965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:17.306965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:08:17.306965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:17.306965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 153/304] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:17.308473 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec699da-f915-41a8-b87b-197d1db8c098 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:17.308473 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec699da-f915-41a8-b87b-197d1db8c098 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:17.308707 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec699da-f915-41a8-b87b-197d1db8c098 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:17.309917 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 154/305] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:17.310617 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec699da-f915-41a8-b87b-197d1db8c098 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:17.325583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea2cb4-5afb-4719-8189-a7e56766cef1 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:17.336428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:17.368816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085ac35-5f25-4a25-aa0c-0394ac17677f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:17.369666 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085ac35-5f25-4a25-aa0c-0394ac17677f None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:17.370091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085ac35-5f25-4a25-aa0c-0394ac17677f None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:17.370642 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085ac35-5f25-4a25-aa0c-0394ac17677f None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:17.371589 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085ac35-5f25-4a25-aa0c-0394ac17677f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:17.372059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085ac35-5f25-4a25-aa0c-0394ac17677f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:17.373695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085ac35-5f25-4a25-aa0c-0394ac17677f None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:17.386022 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 152/306] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:17.388038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef264127-459b-4499-b6de-219b500029d6 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:17.388482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef264127-459b-4499-b6de-219b500029d6 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:17.388901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef264127-459b-4499-b6de-219b500029d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:17.410056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-ef264127-459b-4499-b6de-219b500029d6 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:17.411307 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ef264127-459b-4499-b6de-219b500029d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:17.441953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec699da-f915-41a8-b87b-197d1db8c098 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:17.471634 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:17.491075 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07f809-84c6-4c3b-a7e5-75f312a1d116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:17.492071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07f809-84c6-4c3b-a7e5-75f312a1d116 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:17.492353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07f809-84c6-4c3b-a7e5-75f312a1d116 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:17.492669 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07f809-84c6-4c3b-a7e5-75f312a1d116 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:17.499119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 153/307] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:17.505140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07f809-84c6-4c3b-a7e5-75f312a1d116 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:17.508260 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef264127-459b-4499-b6de-219b500029d6 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:17.510062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-bf07f809-84c6-4c3b-a7e5-75f312a1d116 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:08:17.510062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:17.510062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:17.510062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:17.510062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:17.510062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:17.510062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:17.510062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:17.510062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:17.510062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:17.510062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:17.510062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:17.510062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:17.510062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:17.510062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:17.510062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:17.511242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:17.511242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:17.511242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:17.511242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:17.511242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:17.511242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:17.511242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:17.511242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:17.511242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:17.511242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:17.511242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:17.511242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:17.511242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:17.511242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:17.511242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:17.511242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:17.512129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:17.512129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:17.512129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:17.512129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:17.512129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:17.512129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:17.512129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:17.512129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:17.512129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:17.512129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:17.512129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 21:08:17.512129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:17.512129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 155/308] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:17.512129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07f809-84c6-4c3b-a7e5-75f312a1d116 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:17.611234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:17.640586 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07f809-84c6-4c3b-a7e5-75f312a1d116 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:17.640971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef264127-459b-4499-b6de-219b500029d6 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:17.678585 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba281af-fd56-440b-a8c2-56e84371773d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:17.679211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba281af-fd56-440b-a8c2-56e84371773d None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:17.679503 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba281af-fd56-440b-a8c2-56e84371773d None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:17.679827 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba281af-fd56-440b-a8c2-56e84371773d None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:17.680500 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba281af-fd56-440b-a8c2-56e84371773d None admin] RBAC: Adding query filter params (name=member) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:17.680803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba281af-fd56-440b-a8c2-56e84371773d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:17.682284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba281af-fd56-440b-a8c2-56e84371773d None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:17.694507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb1a20-baf1-4dc6-b425-a7cf82288624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:17.695402 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb1a20-baf1-4dc6-b425-a7cf82288624 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:17.695926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb1a20-baf1-4dc6-b425-a7cf82288624 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:17.696394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb1a20-baf1-4dc6-b425-a7cf82288624 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:17.697368 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 154/309] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:08:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:17.698660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba281af-fd56-440b-a8c2-56e84371773d None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:17.766749 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:17.783372 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb1a20-baf1-4dc6-b425-a7cf82288624 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:17.785279 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb1a20-baf1-4dc6-b425-a7cf82288624 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:17.825471 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba281af-fd56-440b-a8c2-56e84371773d None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:17.878086 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18ed98-2ca8-4693-a168-8bf8c90d0ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:17.879061 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18ed98-2ca8-4693-a168-8bf8c90d0ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:17.879924 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18ed98-2ca8-4693-a168-8bf8c90d0ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:17.880208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18ed98-2ca8-4693-a168-8bf8c90d0ea8 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:17.880877 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18ed98-2ca8-4693-a168-8bf8c90d0ea8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:17.881178 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18ed98-2ca8-4693-a168-8bf8c90d0ea8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:17.882974 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18ed98-2ca8-4693-a168-8bf8c90d0ea8 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:17.886478 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:17.893756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 155/310] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:08:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:17.894620 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18ed98-2ca8-4693-a168-8bf8c90d0ea8 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:17.909928 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 156/311] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:08:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:08:17.912713 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb1a20-baf1-4dc6-b425-a7cf82288624 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:18.033076 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb1a20-baf1-4dc6-b425-a7cf82288624 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:18.035801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18ed98-2ca8-4693-a168-8bf8c90d0ea8 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:18.048508 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:18.081966 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb4c19-a978-4b6e-ad93-9d39b24d07bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:18.084242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb4c19-a978-4b6e-ad93-9d39b24d07bf None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:18.084519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb4c19-a978-4b6e-ad93-9d39b24d07bf None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:18.084709 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47a926-82ed-49d9-8584-3aecf3290c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:18.084817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb4c19-a978-4b6e-ad93-9d39b24d07bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:18.085297 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47a926-82ed-49d9-8584-3aecf3290c23 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:18.085579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47a926-82ed-49d9-8584-3aecf3290c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:18.085853 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47a926-82ed-49d9-8584-3aecf3290c23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:18.097389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb4c19-a978-4b6e-ad93-9d39b24d07bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:18.102934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-fedb4c19-a978-4b6e-ad93-9d39b24d07bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:08:18.102934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:18.102934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:18.102934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:18.102934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:18.102934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:18.102934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:18.102934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:18.102934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:18.102934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:18.102934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:18.102934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:18.102934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:18.102934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:18.102934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:18.102934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:18.103834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:18.103834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:18.103834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:18.103834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:18.103834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:18.103834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:18.103834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:18.103834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:18.103834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:18.103834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:18.103834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:18.103834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:18.103834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:18.103834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:18.103834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:18.103834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:18.104623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:18.104623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:18.104623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:18.104623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:18.104623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:18.104623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:18.104623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:18.104623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:18.104623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:18.104623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:18.104623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:08:18.104623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:18.104623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 157/312] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Oct 21 21:08:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:18.105253 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb4c19-a978-4b6e-ad93-9d39b24d07bf None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:18.160992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:18.186493 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47a926-82ed-49d9-8584-3aecf3290c23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:18.187995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47a926-82ed-49d9-8584-3aecf3290c23 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:18.239131 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb4c19-a978-4b6e-ad93-9d39b24d07bf None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:18.280212 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d83409-9de9-439f-8862-41204fc2e3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:18.281315 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d83409-9de9-439f-8862-41204fc2e3bd None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:18.281601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d83409-9de9-439f-8862-41204fc2e3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:18.281873 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d83409-9de9-439f-8862-41204fc2e3bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:18.326096 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 156/313] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:08:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:08:18.331137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47a926-82ed-49d9-8584-3aecf3290c23 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:18.341574 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:18.382878 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d83409-9de9-439f-8862-41204fc2e3bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:18.384389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d83409-9de9-439f-8862-41204fc2e3bd None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:18.430325 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:18.447485 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47a926-82ed-49d9-8584-3aecf3290c23 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:18.483957 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ce0be-8d32-465e-81af-aae6112a7338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:18.484584 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ce0be-8d32-465e-81af-aae6112a7338 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:18.484859 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ce0be-8d32-465e-81af-aae6112a7338 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:18.485127 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ce0be-8d32-465e-81af-aae6112a7338 None admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:18.486220 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ce0be-8d32-465e-81af-aae6112a7338 None admin] RBAC: Authorizing `identity:get_project(project_id=dd6450bb3add4920a802300916dc915c)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:18.487687 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ce0be-8d32-465e-81af-aae6112a7338 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:18.489335 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 157/314] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:08:18 2022] GET /identity/v3/projects/dd6450bb3add4920a802300916dc915c => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:18.490083 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ce0be-8d32-465e-81af-aae6112a7338 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:18.507450 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 158/315] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:08:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:08:18.507450 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d83409-9de9-439f-8862-41204fc2e3bd None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:18.565777 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:18.586306 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ce0be-8d32-465e-81af-aae6112a7338 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:18.602899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d83409-9de9-439f-8862-41204fc2e3bd None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:18.620145 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c476942-5a32-4477-87f8-a775eca915d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:18.620745 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c476942-5a32-4477-87f8-a775eca915d9 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:18.621025 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c476942-5a32-4477-87f8-a775eca915d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:18.621297 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c476942-5a32-4477-87f8-a775eca915d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:18.621947 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c476942-5a32-4477-87f8-a775eca915d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:18.622250 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c476942-5a32-4477-87f8-a775eca915d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:18.623728 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c476942-5a32-4477-87f8-a775eca915d9 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:18.631931 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 158/316] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:18.633067 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee9772-64d6-46b9-9e9c-1bf9659c981f None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:18.633346 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee9772-64d6-46b9-9e9c-1bf9659c981f None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:18.633684 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee9772-64d6-46b9-9e9c-1bf9659c981f None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:18.635612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 159/317] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:18.635909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee9772-64d6-46b9-9e9c-1bf9659c981f None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:18.640782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf5795-de8a-40ac-9d09-fe29bf91edd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:18.641418 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf5795-de8a-40ac-9d09-fe29bf91edd7 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:18.641696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf5795-de8a-40ac-9d09-fe29bf91edd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:18.641969 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf5795-de8a-40ac-9d09-fe29bf91edd7 None admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:18.643098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf5795-de8a-40ac-9d09-fe29bf91edd7 None admin] RBAC: Authorizing `identity:get_project(project_id=45fcdbb9daa941a89354e8eb723a8fcb)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:18.644559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf5795-de8a-40ac-9d09-fe29bf91edd7 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:18.646246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 159/318] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:08:18 2022] GET /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:18.649051 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf5795-de8a-40ac-9d09-fe29bf91edd7 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:18.659390 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:18.736441 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee9772-64d6-46b9-9e9c-1bf9659c981f None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:18.749593 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf5795-de8a-40ac-9d09-fe29bf91edd7 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:18.769754 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a292f-19c2-420b-8f26-6bcd863c2327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:18.770407 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a292f-19c2-420b-8f26-6bcd863c2327 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:18.770685 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a292f-19c2-420b-8f26-6bcd863c2327 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:18.770963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a292f-19c2-420b-8f26-6bcd863c2327 None admin] PATH_INFO: `/v3/groups/0567312b9f284f2c8cd8a2703553eb5d` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:18.772069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a292f-19c2-420b-8f26-6bcd863c2327 None admin] RBAC: Authorizing `identity:get_group(group_id=0567312b9f284f2c8cd8a2703553eb5d)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:18.773518 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a292f-19c2-420b-8f26-6bcd863c2327 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:18.775088 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 160/319] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Oct 21 21:08:18 2022] GET /identity/v3/groups/0567312b9f284f2c8cd8a2703553eb5d => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:18.775898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a292f-19c2-420b-8f26-6bcd863c2327 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:18.783465 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03487cc5-ee77-4493-aa29-286536696b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:18.784119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03487cc5-ee77-4493-aa29-286536696b04 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:18.784396 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03487cc5-ee77-4493-aa29-286536696b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:18.784666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03487cc5-ee77-4493-aa29-286536696b04 None admin] PATH_INFO: `/v3/projects/dd6450bb3add4920a802300916dc915c` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:18.785785 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03487cc5-ee77-4493-aa29-286536696b04 None admin] RBAC: Authorizing `identity:get_project(project_id=dd6450bb3add4920a802300916dc915c)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:18.787183 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03487cc5-ee77-4493-aa29-286536696b04 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:18.788735 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:18.788872 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 160/320] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:08:18 2022] GET /identity/v3/projects/dd6450bb3add4920a802300916dc915c => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:18.789877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101d6a2-7ab5-42ae-95e4-e61d648796a0 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:18.790178 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101d6a2-7ab5-42ae-95e4-e61d648796a0 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:18.790439 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101d6a2-7ab5-42ae-95e4-e61d648796a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:18.808322 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-8101d6a2-7ab5-42ae-95e4-e61d648796a0 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:18.809481 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8101d6a2-7ab5-42ae-95e4-e61d648796a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:18.875795 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 161/321] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:18.876616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101d6a2-7ab5-42ae-95e4-e61d648796a0 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:18.878028 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a292f-19c2-420b-8f26-6bcd863c2327 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:18.895307 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:18.913054 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d17730-855c-44ee-b580-709f14375354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:18.913669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d17730-855c-44ee-b580-709f14375354 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:18.913952 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d17730-855c-44ee-b580-709f14375354 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:18.914225 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d17730-855c-44ee-b580-709f14375354 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:18.988110 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d17730-855c-44ee-b580-709f14375354 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:18.989961 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d17730-855c-44ee-b580-709f14375354 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:18.993852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101d6a2-7ab5-42ae-95e4-e61d648796a0 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:18.999726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:19.029811 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f48e44-ac39-4101-b67e-6495e1c9bb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:19.030936 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f48e44-ac39-4101-b67e-6495e1c9bb44 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:19.031433 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f48e44-ac39-4101-b67e-6495e1c9bb44 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:19.031942 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f48e44-ac39-4101-b67e-6495e1c9bb44 None admin] PATH_INFO: `/v3/groups/0567312b9f284f2c8cd8a2703553eb5d` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:19.033357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f48e44-ac39-4101-b67e-6495e1c9bb44 None admin] RBAC: Authorizing `identity:get_group(group_id=0567312b9f284f2c8cd8a2703553eb5d)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:19.035186 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f48e44-ac39-4101-b67e-6495e1c9bb44 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:19.037249 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 162/322] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Oct 21 21:08:18 2022] GET /identity/v3/groups/0567312b9f284f2c8cd8a2703553eb5d => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:19.038237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f48e44-ac39-4101-b67e-6495e1c9bb44 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:19.095210 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 161/323] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:08:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:08:19.096356 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d17730-855c-44ee-b580-709f14375354 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:19.120685 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:19.148388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f48e44-ac39-4101-b67e-6495e1c9bb44 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:19.182855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696922ed-1b8b-436c-95db-6e5608bb5708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:19.183840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696922ed-1b8b-436c-95db-6e5608bb5708 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:19.184129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696922ed-1b8b-436c-95db-6e5608bb5708 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:19.184399 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696922ed-1b8b-436c-95db-6e5608bb5708 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:19.185143 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696922ed-1b8b-436c-95db-6e5608bb5708 None admin] RBAC: Adding query filter params (group.id=0567312b9f284f2c8cd8a2703553eb5d, role.id=01dbc90c2ce041afa161e621bd524341, scope.project.id=dd6450bb3add4920a802300916dc915c) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:19.185446 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696922ed-1b8b-436c-95db-6e5608bb5708 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:19.186865 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696922ed-1b8b-436c-95db-6e5608bb5708 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:19.192963 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 163/324] 10.222.0.13 () {56 vars in 1376 bytes} [Fri Oct 21 21:08:19 2022] GET /identity/v3/role_assignments?group.id=0567312b9f284f2c8cd8a2703553eb5d&scope.project.id=dd6450bb3add4920a802300916dc915c&role.id=01dbc90c2ce041afa161e621bd524341 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:19.193075 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d17730-855c-44ee-b580-709f14375354 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:19.193741 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696922ed-1b8b-436c-95db-6e5608bb5708 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:19.213861 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:19.243074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df476ad5-1bfa-469f-928e-9ad88e26e064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:19.243662 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df476ad5-1bfa-469f-928e-9ad88e26e064 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:19.243992 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df476ad5-1bfa-469f-928e-9ad88e26e064 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:19.244263 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df476ad5-1bfa-469f-928e-9ad88e26e064 None admin] PATH_INFO: `/v3/groups/0567312b9f284f2c8cd8a2703553eb5d` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:19.245412 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df476ad5-1bfa-469f-928e-9ad88e26e064 None admin] RBAC: Authorizing `identity:get_group(group_id=0567312b9f284f2c8cd8a2703553eb5d)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:19.246880 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df476ad5-1bfa-469f-928e-9ad88e26e064 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:19.253451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 162/325] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Oct 21 21:08:19 2022] GET /identity/v3/groups/0567312b9f284f2c8cd8a2703553eb5d => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:19.253451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df476ad5-1bfa-469f-928e-9ad88e26e064 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:19.312967 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696922ed-1b8b-436c-95db-6e5608bb5708 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:19.351988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229182b-1f08-49ff-b2ed-6bbdf98565b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:19.352617 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229182b-1f08-49ff-b2ed-6bbdf98565b8 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:19.352895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229182b-1f08-49ff-b2ed-6bbdf98565b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:19.353165 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229182b-1f08-49ff-b2ed-6bbdf98565b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:19.353902 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229182b-1f08-49ff-b2ed-6bbdf98565b8 None admin] RBAC: Adding query filter params (group.id=0567312b9f284f2c8cd8a2703553eb5d, role.id=c73d9ac0c74147e095a9a038d200e2e9, scope.project.id=45fcdbb9daa941a89354e8eb723a8fcb) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:19.354208 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229182b-1f08-49ff-b2ed-6bbdf98565b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:19.355679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229182b-1f08-49ff-b2ed-6bbdf98565b8 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:19.362506 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 164/326] 10.222.0.13 () {56 vars in 1376 bytes} [Fri Oct 21 21:08:19 2022] GET /identity/v3/role_assignments?group.id=0567312b9f284f2c8cd8a2703553eb5d&scope.project.id=45fcdbb9daa941a89354e8eb723a8fcb&role.id=c73d9ac0c74147e095a9a038d200e2e9 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:19.363315 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229182b-1f08-49ff-b2ed-6bbdf98565b8 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:19.365129 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df476ad5-1bfa-469f-928e-9ad88e26e064 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:19.367230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:19.408980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c567b85-b11b-49cb-b34c-68a826b9ad36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:19.409355 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c567b85-b11b-49cb-b34c-68a826b9ad36 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:19.410294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c567b85-b11b-49cb-b34c-68a826b9ad36 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:19.410294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c567b85-b11b-49cb-b34c-68a826b9ad36 None admin] PATH_INFO: `/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:19.411991 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c567b85-b11b-49cb-b34c-68a826b9ad36 None admin] RBAC: Authorizing `identity:get_project(project_id=45fcdbb9daa941a89354e8eb723a8fcb)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:19.413540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c567b85-b11b-49cb-b34c-68a826b9ad36 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:19.415472 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 163/327] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:08:19 2022] GET /identity/v3/projects/45fcdbb9daa941a89354e8eb723a8fcb => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:19.416298 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c567b85-b11b-49cb-b34c-68a826b9ad36 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:19.473422 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:19.507279 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229182b-1f08-49ff-b2ed-6bbdf98565b8 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:19.540658 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c567b85-b11b-49cb-b34c-68a826b9ad36 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:19.550627 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9bd9b-dc57-43d5-bf5b-185bd9db70e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:19.551299 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9bd9b-dc57-43d5-bf5b-185bd9db70e2 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:19.551931 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9bd9b-dc57-43d5-bf5b-185bd9db70e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:19.552213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9bd9b-dc57-43d5-bf5b-185bd9db70e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:19.566936 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9bd9b-dc57-43d5-bf5b-185bd9db70e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:19.573259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-a5d9bd9b-dc57-43d5-bf5b-185bd9db70e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:08:19.573259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:19.573259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:19.573259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:19.573259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:19.573259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:19.573259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:19.573259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:19.573259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:19.573259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:19.573259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:19.573259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:19.573259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:19.573259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:19.573259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:19.573259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:19.574123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:19.574123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:19.574123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:19.574123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:19.574123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:19.574123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:19.574123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:19.574123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:19.574123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:19.574123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:19.574123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:19.574123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:19.574123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:19.574123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:19.574123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:19.574123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:19.574921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:19.574921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:19.574921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:19.574921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:19.574921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:19.574921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:19.574921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:19.574921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:19.574921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:19.574921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:19.574921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:08:19.574921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:19.574921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 165/328] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:08:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:19.575607 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9bd9b-dc57-43d5-bf5b-185bd9db70e2 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:19.577701 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6f29d-857d-4d41-8fbb-41a35633efba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:19.578314 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6f29d-857d-4d41-8fbb-41a35633efba None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:19.578587 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6f29d-857d-4d41-8fbb-41a35633efba None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:19.579697 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6f29d-857d-4d41-8fbb-41a35633efba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:19.580323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f6f29d-857d-4d41-8fbb-41a35633efba None admin] RBAC: Adding query filter params (group.id=0567312b9f284f2c8cd8a2703553eb5d, role.id=c73d9ac0c74147e095a9a038d200e2e9, scope.project.id=dd6450bb3add4920a802300916dc915c) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:19.580638 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f6f29d-857d-4d41-8fbb-41a35633efba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:19.582091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f6f29d-857d-4d41-8fbb-41a35633efba None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:19.589074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 164/329] 10.222.0.13 () {56 vars in 1376 bytes} [Fri Oct 21 21:08:19 2022] GET /identity/v3/role_assignments?group.id=0567312b9f284f2c8cd8a2703553eb5d&scope.project.id=dd6450bb3add4920a802300916dc915c&role.id=c73d9ac0c74147e095a9a038d200e2e9 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:19.589771 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6f29d-857d-4d41-8fbb-41a35633efba None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:19.658438 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:19.705508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9bd9b-dc57-43d5-bf5b-185bd9db70e2 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:19.713474 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6f29d-857d-4d41-8fbb-41a35633efba None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:19.723648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:19.749640 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a772a0c-52b0-4a2a-b275-d127c0aa99c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:19.750285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a772a0c-52b0-4a2a-b275-d127c0aa99c5 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:19.750561 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a772a0c-52b0-4a2a-b275-d127c0aa99c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:19.750829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a772a0c-52b0-4a2a-b275-d127c0aa99c5 None admin] PATH_INFO: `/v3/groups/0567312b9f284f2c8cd8a2703553eb5d` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:19.751997 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a772a0c-52b0-4a2a-b275-d127c0aa99c5 None admin] RBAC: Authorizing `identity:get_group(group_id=0567312b9f284f2c8cd8a2703553eb5d)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:19.753456 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a772a0c-52b0-4a2a-b275-d127c0aa99c5 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:19.755650 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 166/330] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Oct 21 21:08:19 2022] GET /identity/v3/groups/0567312b9f284f2c8cd8a2703553eb5d => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:19.758439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594a1d2-988b-4c60-9055-f10e84d85425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:19.758999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594a1d2-988b-4c60-9055-f10e84d85425 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:19.759280 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594a1d2-988b-4c60-9055-f10e84d85425 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:19.759548 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594a1d2-988b-4c60-9055-f10e84d85425 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:19.760263 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594a1d2-988b-4c60-9055-f10e84d85425 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:19.760563 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594a1d2-988b-4c60-9055-f10e84d85425 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:19.761597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a772a0c-52b0-4a2a-b275-d127c0aa99c5 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:19.761983 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594a1d2-988b-4c60-9055-f10e84d85425 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:19.771511 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 165/331] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:19.777827 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594a1d2-988b-4c60-9055-f10e84d85425 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:19.886571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a772a0c-52b0-4a2a-b275-d127c0aa99c5 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:19.895193 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594a1d2-988b-4c60-9055-f10e84d85425 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:19.920877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:19.928941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a217e-812f-4e07-bb46-f90c67730329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:19.929647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a217e-812f-4e07-bb46-f90c67730329 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:19.930041 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a217e-812f-4e07-bb46-f90c67730329 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:19.930325 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a217e-812f-4e07-bb46-f90c67730329 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:19.931265 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a217e-812f-4e07-bb46-f90c67730329 None admin] RBAC: Adding query filter params (group.id=0567312b9f284f2c8cd8a2703553eb5d, role.id=01dbc90c2ce041afa161e621bd524341, scope.project.id=45fcdbb9daa941a89354e8eb723a8fcb) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:19.931381 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a217e-812f-4e07-bb46-f90c67730329 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:19.932815 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a217e-812f-4e07-bb46-f90c67730329 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:19.935819 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280a06d-9e3b-434f-ba29-dc50ec4bd224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:19.936437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280a06d-9e3b-434f-ba29-dc50ec4bd224 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:19.936718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280a06d-9e3b-434f-ba29-dc50ec4bd224 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:19.936994 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280a06d-9e3b-434f-ba29-dc50ec4bd224 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:19.940285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 167/332] 10.222.0.13 () {56 vars in 1376 bytes} [Fri Oct 21 21:08:19 2022] GET /identity/v3/role_assignments?group.id=0567312b9f284f2c8cd8a2703553eb5d&scope.project.id=45fcdbb9daa941a89354e8eb723a8fcb&role.id=01dbc90c2ce041afa161e621bd524341 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:19.986546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:20.022552 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280a06d-9e3b-434f-ba29-dc50ec4bd224 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:20.024396 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280a06d-9e3b-434f-ba29-dc50ec4bd224 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:20.132887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 166/333] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:08:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 21 21:08:20.138773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a217e-812f-4e07-bb46-f90c67730329 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:20.243921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a217e-812f-4e07-bb46-f90c67730329 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:20.278857 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2144d4-2414-405d-ac79-578ae219bb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:20.279419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2144d4-2414-405d-ac79-578ae219bb0c None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:20.279759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2144d4-2414-405d-ac79-578ae219bb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:20.280037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2144d4-2414-405d-ac79-578ae219bb0c None admin] PATH_INFO: `/v3/projects/64a13af16ef54c01b2a28b4530062691` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:20.281152 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2144d4-2414-405d-ac79-578ae219bb0c None admin] RBAC: Authorizing `identity:get_project(project_id=64a13af16ef54c01b2a28b4530062691)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:20.282604 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2144d4-2414-405d-ac79-578ae219bb0c None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:20.284271 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 168/334] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:08:20 2022] GET /identity/v3/projects/64a13af16ef54c01b2a28b4530062691 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:20.289897 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280a06d-9e3b-434f-ba29-dc50ec4bd224 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:20.361136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:20.392490 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280a06d-9e3b-434f-ba29-dc50ec4bd224 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:20.427411 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46341801-6d27-447e-be83-810e55e3b1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:20.428083 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46341801-6d27-447e-be83-810e55e3b1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:20.428363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46341801-6d27-447e-be83-810e55e3b1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:20.428638 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46341801-6d27-447e-be83-810e55e3b1f6 None admin] PATH_INFO: `/v3/groups/d7b7c44b80564b448289ccfac290425e` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:20.429737 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46341801-6d27-447e-be83-810e55e3b1f6 None admin] RBAC: Authorizing `identity:get_group(group_id=d7b7c44b80564b448289ccfac290425e)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:20.431198 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46341801-6d27-447e-be83-810e55e3b1f6 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:20.432889 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 167/335] 10.222.0.13 () {56 vars in 1180 bytes} [Fri Oct 21 21:08:20 2022] GET /identity/v3/groups/d7b7c44b80564b448289ccfac290425e => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:20.438706 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2144d4-2414-405d-ac79-578ae219bb0c None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:20.497642 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:20.534547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2144d4-2414-405d-ac79-578ae219bb0c None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:20.565659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc86224-4351-4e9a-a8e8-f531f9520c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:20.566249 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc86224-4351-4e9a-a8e8-f531f9520c33 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:20.566524 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc86224-4351-4e9a-a8e8-f531f9520c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:20.567331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc86224-4351-4e9a-a8e8-f531f9520c33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:20.568068 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc86224-4351-4e9a-a8e8-f531f9520c33 None admin] RBAC: Adding query filter params (group.id=d7b7c44b80564b448289ccfac290425e, role.id=c514b065b7604094b899c586f79b0c5b, scope.project.id=64a13af16ef54c01b2a28b4530062691) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:20.568370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc86224-4351-4e9a-a8e8-f531f9520c33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:20.569790 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc86224-4351-4e9a-a8e8-f531f9520c33 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:20.575245 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 169/336] 10.222.0.13 () {56 vars in 1376 bytes} [Fri Oct 21 21:08:20 2022] GET /identity/v3/role_assignments?group.id=d7b7c44b80564b448289ccfac290425e&scope.project.id=64a13af16ef54c01b2a28b4530062691&role.id=c514b065b7604094b899c586f79b0c5b => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:23.187271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d1d91-6530-4fb5-b81c-8fcd4be1ebe5 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:23.187779 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d1d91-6530-4fb5-b81c-8fcd4be1ebe5 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:23.187779 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d1d91-6530-4fb5-b81c-8fcd4be1ebe5 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:23.189568 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 168/337] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:23.195025 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c0ace-0ee8-4aca-a699-383c5e7e9238 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:23.195423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c0ace-0ee8-4aca-a699-383c5e7e9238 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:23.195606 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c0ace-0ee8-4aca-a699-383c5e7e9238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:23.220899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-c55c0ace-0ee8-4aca-a699-383c5e7e9238 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:23.222098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c55c0ace-0ee8-4aca-a699-383c5e7e9238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:23.318064 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 170/338] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:23.481656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe9a4b-2973-4aee-b578-1e38005bfd26 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:23.482170 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe9a4b-2973-4aee-b578-1e38005bfd26 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:23.482589 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe9a4b-2973-4aee-b578-1e38005bfd26 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:23.484347 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 169/339] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:23.490424 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0badee-3959-4e1f-9438-4d8e78ef8112 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:23.490972 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0badee-3959-4e1f-9438-4d8e78ef8112 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:23.491450 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0badee-3959-4e1f-9438-4d8e78ef8112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:23.521007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-aa0badee-3959-4e1f-9438-4d8e78ef8112 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:23.522583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-aa0badee-3959-4e1f-9438-4d8e78ef8112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:23.535075 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226421b-63ac-4242-892c-1df79f762e72 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:23.535709 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226421b-63ac-4242-892c-1df79f762e72 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:23.536194 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226421b-63ac-4242-892c-1df79f762e72 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:23.538079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 170/340] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:23.543298 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f865f7-63d4-4431-b9a1-058ef54e499d None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:23.543832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f865f7-63d4-4431-b9a1-058ef54e499d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:23.544306 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f865f7-63d4-4431-b9a1-058ef54e499d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:23.568832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-59f865f7-63d4-4431-b9a1-058ef54e499d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:23.570787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-59f865f7-63d4-4431-b9a1-058ef54e499d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:23.601719 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:23.631738 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 171/341] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:23.675594 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 171/342] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:08:23.693884 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0badee-3959-4e1f-9438-4d8e78ef8112 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:23.847071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0badee-3959-4e1f-9438-4d8e78ef8112 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:23.882209 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e322b-be72-410f-8b64-3715ed0d64a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:23.882914 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e322b-be72-410f-8b64-3715ed0d64a7 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:23.883196 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e322b-be72-410f-8b64-3715ed0d64a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:23.883467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e322b-be72-410f-8b64-3715ed0d64a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:23.896213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e322b-be72-410f-8b64-3715ed0d64a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:23.897788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e322b-be72-410f-8b64-3715ed0d64a7 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:23.914666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-567e322b-be72-410f-8b64-3715ed0d64a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:23.914666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:23.914666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:23.914666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:23.914666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:23.914666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:23.914666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:23.914666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:23.914666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:23.914666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:23.914666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:23.914666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:23.914666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:23.914666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:23.914666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:23.914666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:23.915692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:23.915692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:23.915692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:23.915692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:23.915692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:23.915692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:23.915692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:23.915692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:23.915692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:23.915692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:23.915692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:23.915692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:23.915692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:23.915692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:23.915692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:23.915692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:23.915692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:23.916551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:23.916551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:23.916551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:23.916551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 172/343] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:23.921988 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f865f7-63d4-4431-b9a1-058ef54e499d None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:23.961924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e322b-be72-410f-8b64-3715ed0d64a7 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:24.031189 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:24.042296 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:24.077150 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f865f7-63d4-4431-b9a1-058ef54e499d None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:24.097416 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e322b-be72-410f-8b64-3715ed0d64a7 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:24.111731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcef123-560f-4079-af71-baaa8291bce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:24.112340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcef123-560f-4079-af71-baaa8291bce1 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:24.112610 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcef123-560f-4079-af71-baaa8291bce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:24.112880 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcef123-560f-4079-af71-baaa8291bce1 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:24.114044 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcef123-560f-4079-af71-baaa8291bce1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:24.114340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcef123-560f-4079-af71-baaa8291bce1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:24.115725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcef123-560f-4079-af71-baaa8291bce1 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:24.126759 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 172/344] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:24.132769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7d2ba-d858-48ad-b2c3-7dea3bc252f6 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:24.133042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf4d88-a4c5-4dbd-90c8-b8f03911df73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:24.133158 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7d2ba-d858-48ad-b2c3-7dea3bc252f6 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:24.133336 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7d2ba-d858-48ad-b2c3-7dea3bc252f6 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:24.133667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf4d88-a4c5-4dbd-90c8-b8f03911df73 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:24.133942 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf4d88-a4c5-4dbd-90c8-b8f03911df73 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:24.134210 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf4d88-a4c5-4dbd-90c8-b8f03911df73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:24.134660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 173/345] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:24.135217 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7d2ba-d858-48ad-b2c3-7dea3bc252f6 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:24.149736 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf4d88-a4c5-4dbd-90c8-b8f03911df73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:24.151167 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf4d88-a4c5-4dbd-90c8-b8f03911df73 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:24.165270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-f0cf4d88-a4c5-4dbd-90c8-b8f03911df73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:24.165270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:24.165270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:24.165270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:24.165270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:24.165270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:24.165270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:24.165270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:24.165270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:24.165270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:24.165270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:24.165270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:24.165270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:24.165270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:24.165270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:24.165270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:24.166192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:24.166192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:24.166192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:24.166192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:24.166192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:24.166192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:24.166192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:24.166192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:24.166192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:24.166192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:24.166192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:24.166192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:24.166192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:24.166192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:24.166192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:24.166192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:24.166192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:24.167030 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:24.167030 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:24.167030 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:24.167030 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 173/346] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:08:24.167237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf4d88-a4c5-4dbd-90c8-b8f03911df73 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:24.251790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7d2ba-d858-48ad-b2c3-7dea3bc252f6 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:24.253986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:24.264883 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf4d88-a4c5-4dbd-90c8-b8f03911df73 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:24.280558 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:24.293510 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f7e52-493f-47e4-a340-45b089a92cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:24.294073 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f7e52-493f-47e4-a340-45b089a92cfe None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:24.294350 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f7e52-493f-47e4-a340-45b089a92cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:24.294619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f7e52-493f-47e4-a340-45b089a92cfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:24.302720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cca95-4217-4f65-8177-572e723be2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:24.303338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cca95-4217-4f65-8177-572e723be2ae None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:24.303634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cca95-4217-4f65-8177-572e723be2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:24.303909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cca95-4217-4f65-8177-572e723be2ae None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:24.304550 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2cca95-4217-4f65-8177-572e723be2ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:24.305978 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2cca95-4217-4f65-8177-572e723be2ae None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:24.306077 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f7e52-493f-47e4-a340-45b089a92cfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:24.307434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f7e52-493f-47e4-a340-45b089a92cfe None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:24.323082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-2e8f7e52-493f-47e4-a340-45b089a92cfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:24.323082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:24.323082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:24.323082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:24.323082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:24.323082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:24.323082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:24.323082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:24.323082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:24.323082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:24.323082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:24.323082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:24.323082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:24.323082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:24.323082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:24.323082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:24.324076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:24.324076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:24.324076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:24.324076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:24.324076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:24.324076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:24.324076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:24.324076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:24.324076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:24.324076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:24.324076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:24.324076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:24.324076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:24.324076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:24.324076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:24.324076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:24.324076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:24.324895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:24.324895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:24.324895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:24.324895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 174/347] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:24.325545 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c36154-b326-476c-a5e6-41e50f488597 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:24.325636 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c36154-b326-476c-a5e6-41e50f488597 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:24.325904 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c36154-b326-476c-a5e6-41e50f488597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:24.344063 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 174/348] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:08:24 2022] POST /identity/v3/users => generated 246 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:08:24.344063 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cca95-4217-4f65-8177-572e723be2ae None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:24.357960 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-f5c36154-b326-476c-a5e6-41e50f488597 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:24.359342 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f5c36154-b326-476c-a5e6-41e50f488597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:24.439042 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 175/349] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:24.444672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c36154-b326-476c-a5e6-41e50f488597 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:24.481056 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cca95-4217-4f65-8177-572e723be2ae None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:24.534951 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4313f-3b75-4c4c-86ba-e0e22889fd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:24.535645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4313f-3b75-4c4c-86ba-e0e22889fd6c None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:24.535925 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4313f-3b75-4c4c-86ba-e0e22889fd6c None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:24.536187 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4313f-3b75-4c4c-86ba-e0e22889fd6c None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:24.536840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4313f-3b75-4c4c-86ba-e0e22889fd6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:24.537140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4313f-3b75-4c4c-86ba-e0e22889fd6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:24.539560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4313f-3b75-4c4c-86ba-e0e22889fd6c None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:24.560691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 175/350] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:24.566106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4313f-3b75-4c4c-86ba-e0e22889fd6c None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:24.577384 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:24.611862 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c36154-b326-476c-a5e6-41e50f488597 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:24.627621 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:24.662011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c0ca1-618b-44fd-ae46-cffc1c4a2932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:24.662632 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c0ca1-618b-44fd-ae46-cffc1c4a2932 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:24.662904 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c0ca1-618b-44fd-ae46-cffc1c4a2932 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:24.663169 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c0ca1-618b-44fd-ae46-cffc1c4a2932 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:24.663833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c0ca1-618b-44fd-ae46-cffc1c4a2932 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:24.664130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c0ca1-618b-44fd-ae46-cffc1c4a2932 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:24.665946 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c0ca1-618b-44fd-ae46-cffc1c4a2932 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:24.679374 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 176/351] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:24.685006 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c0ca1-618b-44fd-ae46-cffc1c4a2932 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:24.693229 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4313f-3b75-4c4c-86ba-e0e22889fd6c None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:24.741196 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25d3ab-f57b-47a8-a0a1-89f880244c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:24.741996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25d3ab-f57b-47a8-a0a1-89f880244c58 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:24.742478 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25d3ab-f57b-47a8-a0a1-89f880244c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:24.742959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25d3ab-f57b-47a8-a0a1-89f880244c58 None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:24.743844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c25d3ab-f57b-47a8-a0a1-89f880244c58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:24.745502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c25d3ab-f57b-47a8-a0a1-89f880244c58 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:24.771781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 176/352] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:08:24 2022] POST /identity/v3/users => generated 248 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:08:24.773696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc589ea2-9e78-47cc-a924-e657a3c87c40 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:24.774440 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc589ea2-9e78-47cc-a924-e657a3c87c40 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:24.775761 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc589ea2-9e78-47cc-a924-e657a3c87c40 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:24.779388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 177/353] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:24.786237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3713914-2eec-441f-9b72-68c207e1e4cc None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:24.786496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3713914-2eec-441f-9b72-68c207e1e4cc None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:24.786760 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3713914-2eec-441f-9b72-68c207e1e4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:24.805701 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-d3713914-2eec-441f-9b72-68c207e1e4cc None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:24.807016 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d3713914-2eec-441f-9b72-68c207e1e4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:24.820112 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c0ca1-618b-44fd-ae46-cffc1c4a2932 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:24.866622 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ccc90-a223-47f6-8d25-465b0444b196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:24.867328 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ccc90-a223-47f6-8d25-465b0444b196 None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:24.867632 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ccc90-a223-47f6-8d25-465b0444b196 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:24.868175 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ccc90-a223-47f6-8d25-465b0444b196 None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:24.868612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ccc90-a223-47f6-8d25-465b0444b196 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:24.870101 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ccc90-a223-47f6-8d25-465b0444b196 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:24.875950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:24.892537 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 178/354] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:08:24.893974 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3713914-2eec-441f-9b72-68c207e1e4cc None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:24.903886 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 177/355] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:08:24 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:08:25.023352 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3713914-2eec-441f-9b72-68c207e1e4cc None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:25.062825 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf91f0-75c7-4175-9924-442b3fbbcc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:25.063889 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf91f0-75c7-4175-9924-442b3fbbcc97 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:25.064393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf91f0-75c7-4175-9924-442b3fbbcc97 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:25.064903 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf91f0-75c7-4175-9924-442b3fbbcc97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:25.077033 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf91f0-75c7-4175-9924-442b3fbbcc97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:25.078943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf91f0-75c7-4175-9924-442b3fbbcc97 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:25.096844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-3bdf91f0-75c7-4175-9924-442b3fbbcc97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:25.096844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:25.096844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:25.096844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:25.096844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:25.096844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:25.096844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:25.096844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:25.096844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:25.096844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:25.096844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:25.096844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:25.096844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:25.096844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:25.096844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:25.096844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:25.098368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:25.098368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:25.098368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:25.098368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:25.098368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:25.098368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:25.098368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:25.098368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:25.098368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:25.098368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:25.098368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:25.098368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:25.098368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:25.098368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:25.098368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:25.098368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:25.098368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:25.099188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:25.099188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:25.099188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:25.101059 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 179/356] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:25.107390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ccc90-a223-47f6-8d25-465b0444b196 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:25.139481 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:25.217698 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ccc90-a223-47f6-8d25-465b0444b196 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:25.239249 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf91f0-75c7-4175-9924-442b3fbbcc97 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:25.263000 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947f437-0d97-49ba-b3b7-f4dcffc39f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:25.263713 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947f437-0d97-49ba-b3b7-f4dcffc39f23 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:25.263927 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947f437-0d97-49ba-b3b7-f4dcffc39f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:25.264453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947f437-0d97-49ba-b3b7-f4dcffc39f23 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:25.265432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947f437-0d97-49ba-b3b7-f4dcffc39f23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:25.265752 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947f437-0d97-49ba-b3b7-f4dcffc39f23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:25.267207 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f947f437-0d97-49ba-b3b7-f4dcffc39f23 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:25.284868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 178/357] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:25.290422 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947f437-0d97-49ba-b3b7-f4dcffc39f23 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:25.302416 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:25.373056 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf91f0-75c7-4175-9924-442b3fbbcc97 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:25.414419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab2888-e57f-47c5-90e7-ae6c1d3a585c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:25.415391 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab2888-e57f-47c5-90e7-ae6c1d3a585c None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:25.416216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:25.417057 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab2888-e57f-47c5-90e7-ae6c1d3a585c None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:25.417522 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab2888-e57f-47c5-90e7-ae6c1d3a585c None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:25.418526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab2888-e57f-47c5-90e7-ae6c1d3a585c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:25.420418 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab2888-e57f-47c5-90e7-ae6c1d3a585c None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:25.430652 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.sql.core [None req-a8ab2888-e57f-47c5-90e7-ae6c1d3a585c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 21 21:08:25.430652 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 21 21:08:25.430652 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [parameters: {'id': '1dfadd110c434312a02896bd71bcbf7e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 21 21:08:25.430652 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81305) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 21 21:08:25.440778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-a8ab2888-e57f-47c5-90e7-ae6c1d3a585c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 21 21:08:25.440778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:25.440778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 21:08:25.440778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 21:08:25.440778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 21:08:25.440778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 21:08:25.440778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 21:08:25.440778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 21:08:25.440778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 21:08:25.440778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application conn.query(q) Oct 21 21:08:25.440778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 21:08:25.440778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 21:08:25.440778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 21:08:25.440778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application result.read() Oct 21 21:08:25.440778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return ref.to_dict() Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 21:08:25.442162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application next(self.gen) Oct 21 21:08:25.443087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 21 21:08:25.443087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application yield resource Oct 21 21:08:25.443087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 21:08:25.443087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application next(self.gen) Oct 21 21:08:25.443087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 21 21:08:25.443087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self.session.rollback() Oct 21 21:08:25.443087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 21 21:08:25.443087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self.force_reraise() Oct 21 21:08:25.443087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 21 21:08:25.443087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise self.value Oct 21 21:08:25.443087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 21 21:08:25.443087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 21 21:08:25.443087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 21 21:08:25.443087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application session.commit() Oct 21 21:08:25.443087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 21 21:08:25.443087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 21 21:08:25.443087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 21 21:08:25.443929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._prepare_impl() Oct 21 21:08:25.443929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 21 21:08:25.443929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self.session.flush() Oct 21 21:08:25.443929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 21 21:08:25.443929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._flush(objects) Oct 21 21:08:25.443929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 21 21:08:25.443929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 21 21:08:25.443929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 21 21:08:25.443929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application compat.raise_( Oct 21 21:08:25.443929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 21:08:25.443929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception Oct 21 21:08:25.443929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 21 21:08:25.443929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application flush_context.execute() Oct 21 21:08:25.443929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 21 21:08:25.443929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rec.execute(self) Oct 21 21:08:25.443929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 21 21:08:25.443929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 21 21:08:25.445720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 21 21:08:25.445720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 21 21:08:25.445720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 21 21:08:25.445720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 21 21:08:25.445720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 21 21:08:25.445720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 21 21:08:25.445720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 21 21:08:25.445720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 21 21:08:25.445720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 21 21:08:25.445720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 21 21:08:25.445720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 21 21:08:25.445720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 21 21:08:25.445720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 21 21:08:25.445720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 21 21:08:25.445720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 21:08:25.445720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception Oct 21 21:08:25.447770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 21:08:25.447770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 21:08:25.447770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 21:08:25.447770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 21:08:25.447770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 21:08:25.447770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 21:08:25.447770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 21:08:25.447770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application conn.query(q) Oct 21 21:08:25.447770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 21:08:25.447770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 21:08:25.447770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 21:08:25.447770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application result.read() Oct 21 21:08:25.447770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 21:08:25.447770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 21:08:25.447770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 21:08:25.447770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 21:08:25.449699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 21:08:25.449699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 21:08:25.449699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 21:08:25.449699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 21:08:25.449699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 21 21:08:25.449699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 21 21:08:25.449699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [parameters: {'id': '1dfadd110c434312a02896bd71bcbf7e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 21 21:08:25.449699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 21 21:08:25.449699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:25.449699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:08:25.449699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:25.449699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:25.449699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:25.449699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:25.449699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:25.449699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:25.451612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:25.451612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:25.451612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:25.451612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:25.451612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:25.451612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:25.451612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 21 21:08:25.451612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 21 21:08:25.451612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:25.451612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:25.451612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 21 21:08:25.451612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 21 21:08:25.451612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 21 21:08:25.451612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 21 21:08:25.451612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 21 21:08:25.451612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:25.455964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 180/358] 10.222.0.13 () {60 vars in 1131 bytes} [Fri Oct 21 21:08:25 2022] POST /identity/v3/roles => generated 144 bytes in 217 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:25.464884 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab2888-e57f-47c5-90e7-ae6c1d3a585c None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:25.467504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947f437-0d97-49ba-b3b7-f4dcffc39f23 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:25.519226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be38a1-a872-40ae-b686-82c2ab5037ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:25.520417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be38a1-a872-40ae-b686-82c2ab5037ad None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:25.520740 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be38a1-a872-40ae-b686-82c2ab5037ad None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:25.521066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be38a1-a872-40ae-b686-82c2ab5037ad None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:25.521695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be38a1-a872-40ae-b686-82c2ab5037ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:25.523254 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be38a1-a872-40ae-b686-82c2ab5037ad None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:25.549108 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 179/359] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:08:25 2022] POST /identity/v3/users => generated 249 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:08:25.633911 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab2888-e57f-47c5-90e7-ae6c1d3a585c None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:25.645663 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:25.673907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d289a-a7c9-44ea-9786-f32e5f787f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:25.674562 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d289a-a7c9-44ea-9786-f32e5f787f2f None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:25.674902 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d289a-a7c9-44ea-9786-f32e5f787f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:25.675181 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d289a-a7c9-44ea-9786-f32e5f787f2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:25.687653 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d289a-a7c9-44ea-9786-f32e5f787f2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:25.693854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-a05d289a-a7c9-44ea-9786-f32e5f787f2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:08:25.693854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:25.693854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:25.693854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:25.693854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:25.693854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:25.693854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:25.693854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:25.693854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:25.693854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:25.693854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:25.693854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:25.693854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:25.693854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:25.693854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:25.693854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:25.694773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:25.694773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:25.694773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:25.694773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:25.694773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:25.694773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:25.694773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:25.694773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:25.694773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:25.694773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:25.694773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:25.694773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:25.694773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:25.694773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:25.694773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:25.694773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:25.695539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:25.695539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:25.695539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:25.695539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:25.695539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:25.695539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:25.695539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:25.695539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:25.695539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:25.695539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:25.695539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 21:08:25.695539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:25.698659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 181/360] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Oct 21 21:08:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:25.701590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be38a1-a872-40ae-b686-82c2ab5037ad None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:25.836927 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be38a1-a872-40ae-b686-82c2ab5037ad None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:25.874115 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ce14d-2db8-4d86-9ef4-7f824fee05c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:25.874736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ce14d-2db8-4d86-9ef4-7f824fee05c4 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:25.875014 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ce14d-2db8-4d86-9ef4-7f824fee05c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:25.875282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ce14d-2db8-4d86-9ef4-7f824fee05c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:25.875991 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ce14d-2db8-4d86-9ef4-7f824fee05c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:25.876292 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ce14d-2db8-4d86-9ef4-7f824fee05c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:25.877663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ce14d-2db8-4d86-9ef4-7f824fee05c4 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:25.889371 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 180/361] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:26.881830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76341cda-eb44-46f0-b1c1-5f1e4922033d None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:26.882396 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76341cda-eb44-46f0-b1c1-5f1e4922033d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:26.882396 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76341cda-eb44-46f0-b1c1-5f1e4922033d None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:26.883970 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 182/362] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:26.889649 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94f9d5-3e88-4c68-8490-445ce16ba09b None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:26.890047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94f9d5-3e88-4c68-8490-445ce16ba09b None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:26.890217 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94f9d5-3e88-4c68-8490-445ce16ba09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:26.901831 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:26.916978 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-bd94f9d5-3e88-4c68-8490-445ce16ba09b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:26.918174 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-bd94f9d5-3e88-4c68-8490-445ce16ba09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:27.001217 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 181/363] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:27.349399 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76341cda-eb44-46f0-b1c1-5f1e4922033d None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:27.453133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76341cda-eb44-46f0-b1c1-5f1e4922033d None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:27.491074 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcdc380-a213-4c99-a2bf-05275d95e051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:27.491739 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdc380-a213-4c99-a2bf-05275d95e051 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:27.492018 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdc380-a213-4c99-a2bf-05275d95e051 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:27.492290 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdc380-a213-4c99-a2bf-05275d95e051 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:27.503039 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcdc380-a213-4c99-a2bf-05275d95e051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:27.508510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-2bcdc380-a213-4c99-a2bf-05275d95e051 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:27.508510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:27.508510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:27.508510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:27.508510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:27.508510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:27.508510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:27.508510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:27.508510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:27.508510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:27.508510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:27.508510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:27.508510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:27.508510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:27.508510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:27.508510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:27.509422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:27.509422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:27.509422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:27.509422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:27.509422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:27.509422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:27.509422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:27.509422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:27.509422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:27.509422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:27.509422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:27.509422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:27.509422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:27.509422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:27.509422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:27.509422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:27.510246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:27.510246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:27.510246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:27.510246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:27.510246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:27.510246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:27.510246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:27.510246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:27.510246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:27.510246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:27.510246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:27.510246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:27.516993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 183/364] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Oct 21 21:08:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:27.522816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94f9d5-3e88-4c68-8490-445ce16ba09b None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:27.675476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94f9d5-3e88-4c68-8490-445ce16ba09b None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:27.722145 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5009f00-4a69-4fee-8bce-26b2cde3048d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:27.725893 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5009f00-4a69-4fee-8bce-26b2cde3048d None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:27.725893 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5009f00-4a69-4fee-8bce-26b2cde3048d None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:27.725893 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5009f00-4a69-4fee-8bce-26b2cde3048d None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:27.725893 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5009f00-4a69-4fee-8bce-26b2cde3048d None admin] RBAC: Adding query filter params (name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:27.725893 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5009f00-4a69-4fee-8bce-26b2cde3048d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:27.726261 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5009f00-4a69-4fee-8bce-26b2cde3048d None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:27.728789 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:27.739678 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 182/365] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Oct 21 21:08:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:27.745284 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcdc380-a213-4c99-a2bf-05275d95e051 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:27.789121 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:27.846665 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811847cb-d55c-40ec-8aac-ac323f68173f None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:27.847297 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811847cb-d55c-40ec-8aac-ac323f68173f None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:27.847873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811847cb-d55c-40ec-8aac-ac323f68173f None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:27.849689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 183/366] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:08:27.855061 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ddc82-11db-4edc-81e7-a740a9522231 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:27.855343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ddc82-11db-4edc-81e7-a740a9522231 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:27.855642 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ddc82-11db-4edc-81e7-a740a9522231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:27.882855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-e38ddc82-11db-4edc-81e7-a740a9522231 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:27.884068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e38ddc82-11db-4edc-81e7-a740a9522231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:27.911302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcdc380-a213-4c99-a2bf-05275d95e051 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:27.956048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbca48e-7038-4e17-8613-6c2f8e280415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:27.956686 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbca48e-7038-4e17-8613-6c2f8e280415 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:27.957502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbca48e-7038-4e17-8613-6c2f8e280415 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:27.957781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbca48e-7038-4e17-8613-6c2f8e280415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:27.968389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbca48e-7038-4e17-8613-6c2f8e280415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:27.969893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbca48e-7038-4e17-8613-6c2f8e280415 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:27.971980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 184/367] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:27.973838 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192c1f1-356e-42f3-ac20-df542964c7cf None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:27.974288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192c1f1-356e-42f3-ac20-df542964c7cf None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:27.974711 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192c1f1-356e-42f3-ac20-df542964c7cf None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:27.976359 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 185/368] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:27.989285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-dfbca48e-7038-4e17-8613-6c2f8e280415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:27.989285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:27.989285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:27.989285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:27.989285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:27.989285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:27.989285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:27.989285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:27.989285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:27.989285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:27.989285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:27.989285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:27.989285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:27.989285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:27.989285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:27.989285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:27.990226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:27.990226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:27.990226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:27.990226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:27.990226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:27.990226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:27.990226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:27.990226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:27.990226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:27.990226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:27.990226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:27.990226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:27.990226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:27.990226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:27.990226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:27.990226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:27.990226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:27.991088 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:27.991088 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:27.991088 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:27.991088 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 184/369] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:27.992136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecf994-6e3b-4b1c-b7ce-71ab69a35e19 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:27.992556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecf994-6e3b-4b1c-b7ce-71ab69a35e19 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:27.992990 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecf994-6e3b-4b1c-b7ce-71ab69a35e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:27.999599 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbca48e-7038-4e17-8613-6c2f8e280415 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:28.044189 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-31ecf994-6e3b-4b1c-b7ce-71ab69a35e19 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:28.045909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-31ecf994-6e3b-4b1c-b7ce-71ab69a35e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:28.084370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:28.115555 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbca48e-7038-4e17-8613-6c2f8e280415 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:28.155277 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c143c36-a03e-444f-8601-6e449d06e4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:28.155944 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c143c36-a03e-444f-8601-6e449d06e4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:28.156222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c143c36-a03e-444f-8601-6e449d06e4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:28.156491 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c143c36-a03e-444f-8601-6e449d06e4c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:28.157142 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c143c36-a03e-444f-8601-6e449d06e4c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:28.157442 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c143c36-a03e-444f-8601-6e449d06e4c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:28.158867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c143c36-a03e-444f-8601-6e449d06e4c5 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:28.165180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 186/370] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:08:28.166717 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8fb9c-2274-48eb-aa5e-635ba0de27d7 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:28.167188 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8fb9c-2274-48eb-aa5e-635ba0de27d7 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:28.167662 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8fb9c-2274-48eb-aa5e-635ba0de27d7 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:28.173779 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 187/371] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:28.175121 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c97efa-d05e-4c48-b831-c5dfb3c5be5d None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:28.175674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c97efa-d05e-4c48-b831-c5dfb3c5be5d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:28.176128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c97efa-d05e-4c48-b831-c5dfb3c5be5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:28.190235 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 185/372] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:28.196351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-88c97efa-d05e-4c48-b831-c5dfb3c5be5d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:28.196500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c143c36-a03e-444f-8601-6e449d06e4c5 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:28.197944 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-88c97efa-d05e-4c48-b831-c5dfb3c5be5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:28.264675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:28.284613 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 188/373] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:28.291607 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c97efa-d05e-4c48-b831-c5dfb3c5be5d None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:28.392232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c143c36-a03e-444f-8601-6e449d06e4c5 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:28.428268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c97efa-d05e-4c48-b831-c5dfb3c5be5d None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:28.451883 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2918b-59a7-43f9-85b4-0c0a89f57b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:28.453023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2918b-59a7-43f9-85b4-0c0a89f57b96 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:28.454829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2918b-59a7-43f9-85b4-0c0a89f57b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:28.455387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2918b-59a7-43f9-85b4-0c0a89f57b96 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:28.468282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2918b-59a7-43f9-85b4-0c0a89f57b96 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:28.470905 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2918b-59a7-43f9-85b4-0c0a89f57b96 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:28.476474 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:28.486504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8315410-30ca-4cef-8fba-595ffdbe7900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:28.487415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8315410-30ca-4cef-8fba-595ffdbe7900 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:28.487910 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8315410-30ca-4cef-8fba-595ffdbe7900 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:28.488441 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8315410-30ca-4cef-8fba-595ffdbe7900 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:28.491890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-fdc2918b-59a7-43f9-85b4-0c0a89f57b96 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 21 21:08:28.491890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:28.491890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:28.491890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:28.491890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:28.491890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:28.491890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:28.491890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:28.491890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:28.491890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:28.491890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:28.491890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:28.491890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:08:28.491890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:08:28.491890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:08:28.491890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:28.492818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:28.493697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:28.493697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:28.493697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:08:28.493697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:08:28.493697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:08:28.493697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:08:28.493697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:08:28.493697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:08:28.493697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 21 21:08:28.493697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:28.497813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 186/374] 10.222.0.13 () {56 vars in 1093 bytes} [Fri Oct 21 21:08:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:28.498704 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2918b-59a7-43f9-85b4-0c0a89f57b96 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:28.504101 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8315410-30ca-4cef-8fba-595ffdbe7900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:28.509352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-c8315410-30ca-4cef-8fba-595ffdbe7900 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:28.509352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:28.509352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:28.509352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:28.509352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:28.509352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:28.509352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:28.509352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:28.509352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:28.509352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:28.509352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:28.509352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:28.509352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:28.509352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:28.509352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:28.509352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:28.510222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:28.510222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:28.510222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:28.510222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:28.510222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:28.510222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:28.510222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:28.510222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:28.510222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:28.510222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:28.510222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:28.510222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:28.510222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:28.510222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:28.510222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:28.510222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:28.511074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:28.511074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:28.511074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:28.511074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:28.511074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:28.511074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:28.511074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:28.511074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:28.511074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:28.511074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:28.511074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:28.511074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:28.513664 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 189/375] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Oct 21 21:08:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:28.515063 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8315410-30ca-4cef-8fba-595ffdbe7900 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:28.569398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:28.609087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2918b-59a7-43f9-85b4-0c0a89f57b96 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:28.623405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8315410-30ca-4cef-8fba-595ffdbe7900 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:28.643383 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c61529-5566-48d3-a605-c801dbe29b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:28.644041 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c61529-5566-48d3-a605-c801dbe29b8a None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:28.644831 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c61529-5566-48d3-a605-c801dbe29b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:28.645104 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c61529-5566-48d3-a605-c801dbe29b8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:28.655362 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c61529-5566-48d3-a605-c801dbe29b8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:28.658714 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af86d95-43de-416d-9b2d-e9eec5dbc36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:28.659319 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af86d95-43de-416d-9b2d-e9eec5dbc36c None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:28.659605 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af86d95-43de-416d-9b2d-e9eec5dbc36c None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:28.659920 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af86d95-43de-416d-9b2d-e9eec5dbc36c None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:28.660657 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af86d95-43de-416d-9b2d-e9eec5dbc36c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:28.660762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-d1c61529-5566-48d3-a605-c801dbe29b8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:28.660762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:28.660762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:28.660762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:28.660762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:28.660762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:28.660762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:28.660762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:28.660762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:28.660762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:28.660762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:28.660762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:28.660762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:28.660762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:28.660762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:28.660762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:28.661569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:28.661569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:28.661569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:28.661569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:28.661569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:28.661569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:28.661569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:28.661569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:28.661569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:28.661569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:28.661569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:28.661569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:28.661569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:28.661569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:28.661569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:28.661569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:28.662427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:28.662427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:28.662427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:28.662427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:28.662427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:28.662427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:28.662427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:28.662427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:28.662427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:28.662427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:28.662427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:28.662427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:28.663022 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af86d95-43de-416d-9b2d-e9eec5dbc36c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:28.663099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 187/376] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Oct 21 21:08:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:28.663099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c61529-5566-48d3-a605-c801dbe29b8a None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:28.667066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af86d95-43de-416d-9b2d-e9eec5dbc36c None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:28.710046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 190/377] 10.222.0.13 () {56 vars in 1133 bytes} [Fri Oct 21 21:08:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:28.715614 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af86d95-43de-416d-9b2d-e9eec5dbc36c None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:28.729912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:28.777345 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c61529-5566-48d3-a605-c801dbe29b8a None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:28.811081 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca54b4f-4a95-466d-887d-22c8e6d07eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:28.811734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca54b4f-4a95-466d-887d-22c8e6d07eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:28.812011 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca54b4f-4a95-466d-887d-22c8e6d07eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:28.812281 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca54b4f-4a95-466d-887d-22c8e6d07eb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:28.812937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca54b4f-4a95-466d-887d-22c8e6d07eb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:28.813237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca54b4f-4a95-466d-887d-22c8e6d07eb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:28.814598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca54b4f-4a95-466d-887d-22c8e6d07eb1 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:28.820756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af86d95-43de-416d-9b2d-e9eec5dbc36c None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:28.822752 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 188/378] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Oct 21 21:08:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:28.823183 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:28.823516 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca54b4f-4a95-466d-887d-22c8e6d07eb1 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:28.856005 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0484e9-19b2-4e35-9698-a244abad937f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:28.856632 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0484e9-19b2-4e35-9698-a244abad937f None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:28.856908 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0484e9-19b2-4e35-9698-a244abad937f None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:28.857176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0484e9-19b2-4e35-9698-a244abad937f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:28.867491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0484e9-19b2-4e35-9698-a244abad937f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:28.872286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-1c0484e9-19b2-4e35-9698-a244abad937f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:28.872286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:28.872286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:28.872286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:28.872286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:28.872286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:28.872286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:28.872286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:28.872286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:28.872286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:28.872286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:28.872286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:28.872286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:28.872286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:28.872286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:28.872286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:28.873216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:28.873216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:28.873216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:28.873216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:28.873216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:28.873216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:28.873216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:28.873216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:28.873216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:28.873216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:28.873216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:28.873216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:28.873216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:28.873216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:28.873216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:28.873216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:28.873989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:28.873989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:28.873989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:28.873989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:28.873989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:28.873989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:28.873989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:28.873989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:28.873989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:28.873989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:28.873989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:28.873989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:28.873989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 191/379] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Oct 21 21:08:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:28.874672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0484e9-19b2-4e35-9698-a244abad937f None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:28.921296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca54b4f-4a95-466d-887d-22c8e6d07eb1 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:28.942992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:28.954233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b99c0-0555-40b6-b7b2-aa3757eb577b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:28.954833 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b99c0-0555-40b6-b7b2-aa3757eb577b None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:28.955106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b99c0-0555-40b6-b7b2-aa3757eb577b None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:28.955373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b99c0-0555-40b6-b7b2-aa3757eb577b None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:28.956050 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b99c0-0555-40b6-b7b2-aa3757eb577b None admin] RBAC: Adding query filter params (name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:28.956346 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b99c0-0555-40b6-b7b2-aa3757eb577b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:28.957754 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b99c0-0555-40b6-b7b2-aa3757eb577b None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:28.965206 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 189/380] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Oct 21 21:08:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:28.966319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2763d93-6436-4db3-9d57-055aac7266c6 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:28.966598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2763d93-6436-4db3-9d57-055aac7266c6 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:28.966865 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2763d93-6436-4db3-9d57-055aac7266c6 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:28.968067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 190/381] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:28.968749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2763d93-6436-4db3-9d57-055aac7266c6 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:28.980387 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0484e9-19b2-4e35-9698-a244abad937f None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:29.014384 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df24127-a0b5-46e2-b3ac-21cfe1d8028c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:29.015013 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df24127-a0b5-46e2-b3ac-21cfe1d8028c None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:29.015295 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df24127-a0b5-46e2-b3ac-21cfe1d8028c None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:29.015590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df24127-a0b5-46e2-b3ac-21cfe1d8028c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:29.026081 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df24127-a0b5-46e2-b3ac-21cfe1d8028c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:29.027633 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df24127-a0b5-46e2-b3ac-21cfe1d8028c None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:29.041672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-9df24127-a0b5-46e2-b3ac-21cfe1d8028c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:29.041672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:29.041672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:29.041672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:29.041672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:29.041672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:29.041672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:29.041672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:29.041672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:29.041672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:29.041672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:29.041672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:29.041672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:29.041672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:29.041672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:29.041672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:29.042738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:29.042738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:29.042738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:29.042738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:29.042738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:29.042738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:29.042738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:29.042738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:29.042738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:29.042738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:29.042738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:29.042738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:29.042738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:29.042738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:29.042738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:29.042738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:29.042738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:29.043558 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:29.043558 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:29.043558 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:29.043558 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 192/382] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:29.043558 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df24127-a0b5-46e2-b3ac-21cfe1d8028c None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:29.061274 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:29.079338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2763d93-6436-4db3-9d57-055aac7266c6 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:29.111650 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d4f42e-fb5b-44a3-8137-5d54caa1b023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:29.112268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4f42e-fb5b-44a3-8137-5d54caa1b023 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:29.112543 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4f42e-fb5b-44a3-8137-5d54caa1b023 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:29.112808 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4f42e-fb5b-44a3-8137-5d54caa1b023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:29.123275 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d4f42e-fb5b-44a3-8137-5d54caa1b023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:29.124743 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d4f42e-fb5b-44a3-8137-5d54caa1b023 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:29.138863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-78d4f42e-fb5b-44a3-8137-5d54caa1b023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:29.138863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:29.138863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:29.138863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:29.138863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:29.138863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:29.138863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:29.138863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:29.138863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:29.138863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:29.138863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:29.138863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:29.138863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:29.138863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:29.138863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:29.138863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:29.140100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:29.140100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:29.140100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:29.140100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:29.140100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:29.140100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:29.140100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:29.140100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:29.140100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:29.140100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:29.140100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:29.140100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:29.140100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:29.140100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:29.140100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:29.140100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:29.140100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:29.140930 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:29.140930 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:29.140930 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:29.140930 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 191/383] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:29.141139 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df24127-a0b5-46e2-b3ac-21cfe1d8028c None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:29.141216 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d4f42e-fb5b-44a3-8137-5d54caa1b023 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:29.174117 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:29.175447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f09109-b0fe-4d1a-9d97-7acf68701b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:29.176791 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f09109-b0fe-4d1a-9d97-7acf68701b37 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:29.177070 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f09109-b0fe-4d1a-9d97-7acf68701b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:29.177341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f09109-b0fe-4d1a-9d97-7acf68701b37 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:29.177993 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f09109-b0fe-4d1a-9d97-7acf68701b37 None admin] RBAC: Adding query filter params (name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:29.178293 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f09109-b0fe-4d1a-9d97-7acf68701b37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:29.179697 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f09109-b0fe-4d1a-9d97-7acf68701b37 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:29.187400 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 193/384] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Oct 21 21:08:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:29.188744 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80e806-325b-4558-87b4-a5b3da821069 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:29.189022 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80e806-325b-4558-87b4-a5b3da821069 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:29.189350 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80e806-325b-4558-87b4-a5b3da821069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:29.206657 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-cf80e806-325b-4558-87b4-a5b3da821069 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:29.207899 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-cf80e806-325b-4558-87b4-a5b3da821069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:29.238606 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d4f42e-fb5b-44a3-8137-5d54caa1b023 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:29.270718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856d6b7-73a4-48bb-99a4-551126bad55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:29.270868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 194/385] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:29.271333 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856d6b7-73a4-48bb-99a4-551126bad55b None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:29.271627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80e806-325b-4558-87b4-a5b3da821069 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:29.271739 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856d6b7-73a4-48bb-99a4-551126bad55b None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:29.272045 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856d6b7-73a4-48bb-99a4-551126bad55b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:29.281136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:29.293610 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856d6b7-73a4-48bb-99a4-551126bad55b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:29.295223 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856d6b7-73a4-48bb-99a4-551126bad55b None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:29.309634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-3856d6b7-73a4-48bb-99a4-551126bad55b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:29.309634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:29.309634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:29.309634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:29.309634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:29.309634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:29.309634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:29.309634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:29.309634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:29.309634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:29.309634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:29.309634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:29.309634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:29.309634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:29.309634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:29.309634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:29.310526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:29.310526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:29.310526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:29.310526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:29.310526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:29.310526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:29.310526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:29.310526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:29.310526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:29.310526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:29.310526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:29.310526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:29.310526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:29.310526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:29.310526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:29.310526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:29.310526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:29.316252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:29.316252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:29.316252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:29.316252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 192/386] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:29.316252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856d6b7-73a4-48bb-99a4-551126bad55b None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:29.378660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80e806-325b-4558-87b4-a5b3da821069 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:29.409600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:29.416309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987f992-338c-445f-aae6-c5cd3443e900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:29.417127 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987f992-338c-445f-aae6-c5cd3443e900 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:29.417416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987f992-338c-445f-aae6-c5cd3443e900 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:29.417685 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987f992-338c-445f-aae6-c5cd3443e900 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:29.418323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987f992-338c-445f-aae6-c5cd3443e900 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:29.418626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987f992-338c-445f-aae6-c5cd3443e900 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:29.420055 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987f992-338c-445f-aae6-c5cd3443e900 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:29.423836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856d6b7-73a4-48bb-99a4-551126bad55b None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:29.436168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 195/387] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:29.436168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987f992-338c-445f-aae6-c5cd3443e900 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:29.463814 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca1001-0cc9-4cfc-9e7f-9ef4db76ec38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:29.464420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca1001-0cc9-4cfc-9e7f-9ef4db76ec38 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:29.464699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca1001-0cc9-4cfc-9e7f-9ef4db76ec38 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:29.464970 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca1001-0cc9-4cfc-9e7f-9ef4db76ec38 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:29.465597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca1001-0cc9-4cfc-9e7f-9ef4db76ec38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:29.465921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca1001-0cc9-4cfc-9e7f-9ef4db76ec38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:29.467245 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca1001-0cc9-4cfc-9e7f-9ef4db76ec38 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:29.483255 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 193/388] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:29.483255 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca1001-0cc9-4cfc-9e7f-9ef4db76ec38 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:29.507661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:29.544503 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987f992-338c-445f-aae6-c5cd3443e900 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:29.577599 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca08d9-49df-4c62-bf99-dc85cd42599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:29.578252 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca08d9-49df-4c62-bf99-dc85cd42599d None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:29.578544 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca08d9-49df-4c62-bf99-dc85cd42599d None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:29.578794 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca08d9-49df-4c62-bf99-dc85cd42599d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:29.581898 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca1001-0cc9-4cfc-9e7f-9ef4db76ec38 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:29.589829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca08d9-49df-4c62-bf99-dc85cd42599d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:29.591293 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca08d9-49df-4c62-bf99-dc85cd42599d None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:29.605626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-e5ca08d9-49df-4c62-bf99-dc85cd42599d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:29.605626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:29.605626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:29.605626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:29.605626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:29.605626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:29.605626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:29.605626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:29.605626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:29.605626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:29.605626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:29.605626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:29.605626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:29.605626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:29.605626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:29.605626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:29.606661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:29.606661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:29.606661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:29.606661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:29.606661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:29.606661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:29.606661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:29.606661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:29.606661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:29.606661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:29.606661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:29.606661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:29.606661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:29.606661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:29.606661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:29.606661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:29.606661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:29.607479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:29.607479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:29.607479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:29.607479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 196/389] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:29.607742 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca08d9-49df-4c62-bf99-dc85cd42599d None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:29.616805 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664b6a0-3711-48b7-8bb8-c69d10a4bef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:29.617408 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664b6a0-3711-48b7-8bb8-c69d10a4bef0 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:29.617682 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664b6a0-3711-48b7-8bb8-c69d10a4bef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:29.617953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664b6a0-3711-48b7-8bb8-c69d10a4bef0 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:29.618582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664b6a0-3711-48b7-8bb8-c69d10a4bef0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:29.618882 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664b6a0-3711-48b7-8bb8-c69d10a4bef0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:29.620269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664b6a0-3711-48b7-8bb8-c69d10a4bef0 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:29.631110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 194/390] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:29.631902 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664b6a0-3711-48b7-8bb8-c69d10a4bef0 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:29.643554 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:29.710314 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca08d9-49df-4c62-bf99-dc85cd42599d None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:29.729805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:29.732106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664b6a0-3711-48b7-8bb8-c69d10a4bef0 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:29.743901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5ebb7-0708-49f6-9219-8ab3f0a49a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:29.744526 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5ebb7-0708-49f6-9219-8ab3f0a49a3d None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:29.744803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5ebb7-0708-49f6-9219-8ab3f0a49a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:29.745074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5ebb7-0708-49f6-9219-8ab3f0a49a3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:29.751718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5ebb7-0708-49f6-9219-8ab3f0a49a3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:29.753169 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5ebb7-0708-49f6-9219-8ab3f0a49a3d None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:29.765410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-90d5ebb7-0708-49f6-9219-8ab3f0a49a3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:29.765410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:29.765410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:29.765410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:29.765410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:29.765410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:29.765410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:29.765410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:29.765410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:29.765410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:29.765410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:29.765410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:29.765410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:08:29.765410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:08:29.765410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:08:29.766268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:08:29.766268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:29.766268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:29.766268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:29.766268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:29.766268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:29.766268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:29.766268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:29.766268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:29.766268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:29.766268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:29.766268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:29.766268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:29.766268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:29.766268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:29.766268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:29.766268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:29.767143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:08:29.767143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:08:29.767143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:08:29.767143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:08:29.767143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:08:29.767143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:08:29.767143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:29.767143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:29.767143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 197/391] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:29.767652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5ebb7-0708-49f6-9219-8ab3f0a49a3d None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:29.772249 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c3b59-28b0-4462-8e8e-4914559d06fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:29.772887 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c3b59-28b0-4462-8e8e-4914559d06fc None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:29.773165 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c3b59-28b0-4462-8e8e-4914559d06fc None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:29.773434 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c3b59-28b0-4462-8e8e-4914559d06fc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:29.781687 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c3b59-28b0-4462-8e8e-4914559d06fc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:29.783199 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c3b59-28b0-4462-8e8e-4914559d06fc None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:29.794172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-8d1c3b59-28b0-4462-8e8e-4914559d06fc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 21 21:08:29.794172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:29.794172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:29.794172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:29.794172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:29.794172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:29.794172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:29.794172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:29.794172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:29.794172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:29.794172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:29.794172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:29.794172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:08:29.794172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:08:29.794172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:08:29.794172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:29.795190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:29.796094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:29.796094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:29.796094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:08:29.796094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:08:29.796094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:08:29.796094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:08:29.796094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:08:29.796094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:08:29.796094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 21 21:08:29.796094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:29.796094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 195/392] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:29.796094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c3b59-28b0-4462-8e8e-4914559d06fc None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:29.859446 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:29.871139 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5ebb7-0708-49f6-9219-8ab3f0a49a3d None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:29.895272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c3b59-28b0-4462-8e8e-4914559d06fc None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:29.904016 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06703b-4db5-4f18-b758-c730a7cd9b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:29.904637 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06703b-4db5-4f18-b758-c730a7cd9b63 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:29.904916 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06703b-4db5-4f18-b758-c730a7cd9b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:29.905190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06703b-4db5-4f18-b758-c730a7cd9b63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:29.916084 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06703b-4db5-4f18-b758-c730a7cd9b63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:29.917541 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06703b-4db5-4f18-b758-c730a7cd9b63 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:29.927446 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00f5a5-7d09-421e-813a-284308cccf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:29.928062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00f5a5-7d09-421e-813a-284308cccf03 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:29.928343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00f5a5-7d09-421e-813a-284308cccf03 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:29.928617 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00f5a5-7d09-421e-813a-284308cccf03 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:29.929278 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00f5a5-7d09-421e-813a-284308cccf03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:29.929582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00f5a5-7d09-421e-813a-284308cccf03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:29.930929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00f5a5-7d09-421e-813a-284308cccf03 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:29.931275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-4f06703b-4db5-4f18-b758-c730a7cd9b63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:29.931275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:29.931275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:29.931275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:29.931275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:29.931275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:29.931275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:29.931275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:29.931275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:29.931275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:29.931275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:29.931275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:29.931275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:29.931275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:29.931275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:29.931275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:29.932196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:29.932196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:29.932196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:29.932196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:29.932196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:29.932196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:29.932196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:29.932196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:29.932196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:29.932196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:29.932196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:29.932196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:29.932196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:29.932196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:29.932196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:29.932196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:29.932196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:29.933040 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:29.933040 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:29.933040 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:29.933040 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 198/393] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:29.933281 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06703b-4db5-4f18-b758-c730a7cd9b63 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:29.945697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 196/394] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:29.946501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00f5a5-7d09-421e-813a-284308cccf03 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:29.966946 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:30.034082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06703b-4db5-4f18-b758-c730a7cd9b63 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:30.042667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00f5a5-7d09-421e-813a-284308cccf03 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:30.066747 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00317540-3081-4cf9-a736-ec0cc76651b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:30.067302 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00317540-3081-4cf9-a736-ec0cc76651b8 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:30.067592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00317540-3081-4cf9-a736-ec0cc76651b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:30.067868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00317540-3081-4cf9-a736-ec0cc76651b8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:30.068287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:30.075407 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00317540-3081-4cf9-a736-ec0cc76651b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:30.076269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8203403-659c-43e6-9dc9-5b513a63906c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:30.076870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8203403-659c-43e6-9dc9-5b513a63906c None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:30.076988 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00317540-3081-4cf9-a736-ec0cc76651b8 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:30.077151 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8203403-659c-43e6-9dc9-5b513a63906c None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:30.077512 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8203403-659c-43e6-9dc9-5b513a63906c None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:30.078197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8203403-659c-43e6-9dc9-5b513a63906c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:30.078503 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8203403-659c-43e6-9dc9-5b513a63906c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:30.079939 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8203403-659c-43e6-9dc9-5b513a63906c None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:30.089437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-00317540-3081-4cf9-a736-ec0cc76651b8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 21 21:08:30.089437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:30.089437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:30.089437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:30.089437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:30.089437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:30.089437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:30.089437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:30.089437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:30.089437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:30.089437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:30.089437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:30.089437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:08:30.089437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:08:30.089437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:08:30.089437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:30.090380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:30.091282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:30.091282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:30.091282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:08:30.091282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:08:30.091282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:08:30.091282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:08:30.091282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:08:30.091282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:08:30.091282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 21 21:08:30.091282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:30.091282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 199/395] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:30.091905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00317540-3081-4cf9-a736-ec0cc76651b8 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:30.099411 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 197/396] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:08:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:30.100252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8203403-659c-43e6-9dc9-5b513a63906c None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:30.183557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:30.192529 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00317540-3081-4cf9-a736-ec0cc76651b8 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:30.196711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8203403-659c-43e6-9dc9-5b513a63906c None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:30.225122 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e3b76-2570-4009-80a3-95fbcb52997c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:30.225732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e3b76-2570-4009-80a3-95fbcb52997c None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:30.226009 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e3b76-2570-4009-80a3-95fbcb52997c None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:30.226279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e3b76-2570-4009-80a3-95fbcb52997c None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:30.227008 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e3b76-2570-4009-80a3-95fbcb52997c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:30.227311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e3b76-2570-4009-80a3-95fbcb52997c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:30.228782 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e3b76-2570-4009-80a3-95fbcb52997c None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:30.229750 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb7bdf-b1d0-4506-9253-a3cd95412d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:30.230344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb7bdf-b1d0-4506-9253-a3cd95412d4f None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:30.230718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb7bdf-b1d0-4506-9253-a3cd95412d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:30.230992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb7bdf-b1d0-4506-9253-a3cd95412d4f None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:30.231646 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb7bdf-b1d0-4506-9253-a3cd95412d4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:30.231942 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb7bdf-b1d0-4506-9253-a3cd95412d4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:30.233276 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb7bdf-b1d0-4506-9253-a3cd95412d4f None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:30.244094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 198/397] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:30.244903 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb7bdf-b1d0-4506-9253-a3cd95412d4f None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:30.246457 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 200/398] 10.222.0.13 () {56 vars in 1137 bytes} [Fri Oct 21 21:08:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:30.247188 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e3b76-2570-4009-80a3-95fbcb52997c None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:30.295241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:30.342818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e3b76-2570-4009-80a3-95fbcb52997c None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:30.345285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb7bdf-b1d0-4506-9253-a3cd95412d4f None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:30.375323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccbf74-092f-4890-a574-af2511182d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:30.375943 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccbf74-092f-4890-a574-af2511182d70 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:30.376217 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccbf74-092f-4890-a574-af2511182d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:30.376485 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccbf74-092f-4890-a574-af2511182d70 None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:30.377200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bccbf74-092f-4890-a574-af2511182d70 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:30.377505 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bccbf74-092f-4890-a574-af2511182d70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:30.378296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812d409-12ef-4dfd-bf57-80312505e0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:30.379410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812d409-12ef-4dfd-bf57-80312505e0be None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:30.379504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bccbf74-092f-4890-a574-af2511182d70 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:30.379719 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812d409-12ef-4dfd-bf57-80312505e0be None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:30.380013 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812d409-12ef-4dfd-bf57-80312505e0be None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:30.387320 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812d409-12ef-4dfd-bf57-80312505e0be None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:30.388780 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812d409-12ef-4dfd-bf57-80312505e0be None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:30.395976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 201/399] 10.222.0.13 () {56 vars in 1137 bytes} [Fri Oct 21 21:08:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:30.396740 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccbf74-092f-4890-a574-af2511182d70 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:30.400489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-b812d409-12ef-4dfd-bf57-80312505e0be None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 21 21:08:30.400489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:30.400489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:30.400489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:30.400489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:30.400489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:30.400489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:30.400489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:30.400489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:30.400489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:30.400489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:30.400489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:30.400489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:08:30.400489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:08:30.400489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:08:30.400489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:30.401486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:30.402723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:30.402723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:30.402723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:08:30.402723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:08:30.402723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:08:30.402723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:08:30.402723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:08:30.402723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:08:30.402723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 21 21:08:30.402723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:30.402723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 199/400] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Oct 21 21:08:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:30.402723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812d409-12ef-4dfd-bf57-80312505e0be None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:30.407496 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:30.493031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccbf74-092f-4890-a574-af2511182d70 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:30.500280 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812d409-12ef-4dfd-bf57-80312505e0be None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:30.509214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:30.525326 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa90dc-6692-4832-bd32-c0286b7b5d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:30.525972 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa90dc-6692-4832-bd32-c0286b7b5d47 None admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:30.526245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa90dc-6692-4832-bd32-c0286b7b5d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:30.526517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa90dc-6692-4832-bd32-c0286b7b5d47 None admin] PATH_INFO: `/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/12dad9f016a740b98538722e1b723823/roles/b0f9a090148d4a0eb6456472edec3d5a` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:30.533662 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d7ef6-b7b5-45ae-a162-2db94e18afb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:30.534259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d7ef6-b7b5-45ae-a162-2db94e18afb2 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:30.534537 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d7ef6-b7b5-45ae-a162-2db94e18afb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:30.535300 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d7ef6-b7b5-45ae-a162-2db94e18afb2 None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:30.535970 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d7ef6-b7b5-45ae-a162-2db94e18afb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:30.537392 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d7ef6-b7b5-45ae-a162-2db94e18afb2 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:30.540564 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa90dc-6692-4832-bd32-c0286b7b5d47 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b2c33250ef84d79a4a4742fb4252156, user_id=12dad9f016a740b98538722e1b723823, role_id=b0f9a090148d4a0eb6456472edec3d5a)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:30.541974 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa90dc-6692-4832-bd32-c0286b7b5d47 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:30.554549 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 202/401] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:08:30 2022] PUT /identity/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/12dad9f016a740b98538722e1b723823/roles/b0f9a090148d4a0eb6456472edec3d5a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:08:30.555329 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa90dc-6692-4832-bd32-c0286b7b5d47 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:30.556742 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 200/402] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:08:30 2022] POST /identity/v3/users => generated 247 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:08:30.557624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d7ef6-b7b5-45ae-a162-2db94e18afb2 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:30.629589 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:30.679499 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa90dc-6692-4832-bd32-c0286b7b5d47 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:30.700466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d7ef6-b7b5-45ae-a162-2db94e18afb2 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:30.720251 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300e276-51c2-4dc8-acc0-8eeb3e15421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:30.720920 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300e276-51c2-4dc8-acc0-8eeb3e15421d None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:30.721203 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300e276-51c2-4dc8-acc0-8eeb3e15421d None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:30.724912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300e276-51c2-4dc8-acc0-8eeb3e15421d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:30.737323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300e276-51c2-4dc8-acc0-8eeb3e15421d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:30.738513 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300e276-51c2-4dc8-acc0-8eeb3e15421d None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:30.751544 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737130fd-954c-4967-b5d5-5045cb1ab222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:30.752214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737130fd-954c-4967-b5d5-5045cb1ab222 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:30.752495 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737130fd-954c-4967-b5d5-5045cb1ab222 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:30.752765 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737130fd-954c-4967-b5d5-5045cb1ab222 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:30.759286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-8300e276-51c2-4dc8-acc0-8eeb3e15421d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:30.759286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:30.759286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:30.759286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:30.759286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:30.759286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:30.759286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:30.759286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:30.759286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:30.759286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:30.759286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:30.759286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:30.759286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:30.759286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:30.759286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:30.759286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:30.760223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:30.760223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:30.760223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:30.760223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:30.760223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:30.760223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:30.760223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:30.760223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:30.760223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:30.760223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:30.760223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:30.760223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:30.760223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:30.760223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:30.760223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:30.760223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:30.760223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:30.761067 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:30.761067 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:30.761067 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:30.761067 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 203/403] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:30.761551 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300e276-51c2-4dc8-acc0-8eeb3e15421d None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:30.773040 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737130fd-954c-4967-b5d5-5045cb1ab222 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:30.775222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737130fd-954c-4967-b5d5-5045cb1ab222 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:30.792507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-737130fd-954c-4967-b5d5-5045cb1ab222 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:30.792507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:30.792507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:30.792507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:30.792507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:30.792507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:30.792507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:30.792507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:30.792507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:30.792507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:30.792507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:30.792507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:30.792507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:30.792507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:30.792507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:30.792507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:30.793443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:30.793443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:30.793443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:30.793443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:30.793443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:30.793443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:30.793443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:30.793443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:30.793443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:30.793443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:30.793443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:30.793443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:30.793443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:30.793443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:30.793443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:30.793443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:30.793443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:30.794269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:30.794269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:30.794269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:30.794269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 201/404] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:30.794555 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737130fd-954c-4967-b5d5-5045cb1ab222 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:30.816358 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:30.884483 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300e276-51c2-4dc8-acc0-8eeb3e15421d None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:30.895938 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:30.920236 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468dd8b3-3849-41aa-a847-5344b99a18f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:30.920848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468dd8b3-3849-41aa-a847-5344b99a18f0 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:30.921125 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468dd8b3-3849-41aa-a847-5344b99a18f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:30.921395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468dd8b3-3849-41aa-a847-5344b99a18f0 None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:30.922121 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468dd8b3-3849-41aa-a847-5344b99a18f0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:30.922428 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468dd8b3-3849-41aa-a847-5344b99a18f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:30.922734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737130fd-954c-4967-b5d5-5045cb1ab222 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:30.923966 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468dd8b3-3849-41aa-a847-5344b99a18f0 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:30.943204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 204/405] 10.222.0.13 () {56 vars in 1139 bytes} [Fri Oct 21 21:08:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:30.944084 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468dd8b3-3849-41aa-a847-5344b99a18f0 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:30.961641 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf9585-bed1-4dbe-b6be-d2094bf461a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:30.963004 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf9585-bed1-4dbe-b6be-d2094bf461a3 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:30.963551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf9585-bed1-4dbe-b6be-d2094bf461a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:30.964029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf9585-bed1-4dbe-b6be-d2094bf461a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:30.964890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf9585-bed1-4dbe-b6be-d2094bf461a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:30.965364 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf9585-bed1-4dbe-b6be-d2094bf461a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:30.967069 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf9585-bed1-4dbe-b6be-d2094bf461a3 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:30.980746 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 202/406] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:30.981837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf9585-bed1-4dbe-b6be-d2094bf461a3 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:31.055038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468dd8b3-3849-41aa-a847-5344b99a18f0 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:31.071931 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:31.090108 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71957f90-982f-4a17-86b7-685ede5e374c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:31.090748 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71957f90-982f-4a17-86b7-685ede5e374c None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:31.091025 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71957f90-982f-4a17-86b7-685ede5e374c None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:31.091297 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71957f90-982f-4a17-86b7-685ede5e374c None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:31.092100 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71957f90-982f-4a17-86b7-685ede5e374c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:31.092408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71957f90-982f-4a17-86b7-685ede5e374c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:31.093860 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71957f90-982f-4a17-86b7-685ede5e374c None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:31.098310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf9585-bed1-4dbe-b6be-d2094bf461a3 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:31.105291 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 205/407] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:31.110459 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71957f90-982f-4a17-86b7-685ede5e374c None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:31.140024 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:31.141414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bc27b-c31e-4a68-b211-0214708f0e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:31.142056 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bc27b-c31e-4a68-b211-0214708f0e91 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:31.142340 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bc27b-c31e-4a68-b211-0214708f0e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:31.142610 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bc27b-c31e-4a68-b211-0214708f0e91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:31.153288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bc27b-c31e-4a68-b211-0214708f0e91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:31.154869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bc27b-c31e-4a68-b211-0214708f0e91 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:31.170042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-0f7bc27b-c31e-4a68-b211-0214708f0e91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:31.170042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:31.170042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:31.170042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:31.170042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:31.170042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:31.170042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:31.170042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:31.170042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:31.170042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:31.170042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:31.170042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:31.170042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:31.170042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:31.170042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:31.170042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:31.171129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:31.171129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:31.171129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:31.171129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:31.171129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:31.171129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:31.171129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:31.171129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:31.171129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:31.171129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:31.171129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:31.171129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:31.171129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:31.171129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:31.171129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:31.171129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:31.171129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:31.172193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:31.172193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:31.172193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:31.172193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 203/408] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:31.176629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bc27b-c31e-4a68-b211-0214708f0e91 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:31.234476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71957f90-982f-4a17-86b7-685ede5e374c None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:31.269783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2b172-cbcf-425b-a92f-52375cce871e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:31.270419 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2b172-cbcf-425b-a92f-52375cce871e None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:31.270696 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2b172-cbcf-425b-a92f-52375cce871e None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:31.270964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2b172-cbcf-425b-a92f-52375cce871e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:31.278118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2b172-cbcf-425b-a92f-52375cce871e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:31.279598 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2b172-cbcf-425b-a92f-52375cce871e None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:31.292002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-9bd2b172-cbcf-425b-a92f-52375cce871e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:31.292002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:31.292002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:31.292002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:31.292002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:31.292002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:31.292002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:31.292002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:31.292002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:31.292002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:31.292002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:31.292002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:31.292002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:08:31.292002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:08:31.292002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:08:31.293031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:08:31.293031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:31.293031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:31.293031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:31.293031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:31.293031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:31.293031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:31.293031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:31.293031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:31.293031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:31.293031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:31.293031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:31.293031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:31.293031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:31.293031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:31.293031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:31.293031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:31.294059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:08:31.294059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:08:31.294059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:08:31.294059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:08:31.294059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:08:31.294059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:08:31.294059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:31.294059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:31.294059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 206/409] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:31.296553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bc27b-c31e-4a68-b211-0214708f0e91 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:31.298433 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2b172-cbcf-425b-a92f-52375cce871e None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:31.338854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5b744-3f91-41a1-8331-66da80a2da2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:31.339621 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5b744-3f91-41a1-8331-66da80a2da2c None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:31.340337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5b744-3f91-41a1-8331-66da80a2da2c None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:31.340619 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5b744-3f91-41a1-8331-66da80a2da2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:31.350585 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5b744-3f91-41a1-8331-66da80a2da2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:31.351144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:31.356642 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5b744-3f91-41a1-8331-66da80a2da2c None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:31.374778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-6ea5b744-3f91-41a1-8331-66da80a2da2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:31.374778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:31.374778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:31.374778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:31.374778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:31.374778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:31.374778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:31.374778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:31.374778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:31.374778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:31.374778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:31.374778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:31.374778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:08:31.374778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:08:31.374778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:08:31.375696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:08:31.375696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:31.375696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:31.375696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:31.375696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:31.375696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:31.375696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:31.375696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:31.375696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:31.375696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:31.375696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:31.375696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:31.375696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:31.375696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:31.375696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:31.375696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:31.375696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:31.376514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:08:31.376514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:08:31.376514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:08:31.376514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:08:31.376514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:08:31.376514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:08:31.376514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:31.376514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:31.376897 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 204/410] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:31.382883 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5b744-3f91-41a1-8331-66da80a2da2c None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:31.398890 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:31.416665 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2b172-cbcf-425b-a92f-52375cce871e None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:31.461194 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec9299-5337-47b7-86b0-86d3b71de923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:31.462068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec9299-5337-47b7-86b0-86d3b71de923 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:31.462500 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec9299-5337-47b7-86b0-86d3b71de923 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:31.463333 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec9299-5337-47b7-86b0-86d3b71de923 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:31.464144 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec9299-5337-47b7-86b0-86d3b71de923 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:31.464590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec9299-5337-47b7-86b0-86d3b71de923 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:31.466483 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec9299-5337-47b7-86b0-86d3b71de923 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:31.482807 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 207/411] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:31.484023 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec9299-5337-47b7-86b0-86d3b71de923 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:31.495893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5b744-3f91-41a1-8331-66da80a2da2c None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:31.534600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657123c-ffb4-49ca-921f-667b8dbb80cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:31.535281 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657123c-ffb4-49ca-921f-667b8dbb80cf None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:31.535561 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657123c-ffb4-49ca-921f-667b8dbb80cf None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:31.535870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657123c-ffb4-49ca-921f-667b8dbb80cf None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:31.536566 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657123c-ffb4-49ca-921f-667b8dbb80cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:31.536873 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657123c-ffb4-49ca-921f-667b8dbb80cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:31.538288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657123c-ffb4-49ca-921f-667b8dbb80cf None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:31.551701 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 205/412] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:08:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:31.556508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657123c-ffb4-49ca-921f-667b8dbb80cf None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:31.607733 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec9299-5337-47b7-86b0-86d3b71de923 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:31.623424 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:31.642726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a34f0-603f-437a-acaf-8f22d86e7bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:31.643524 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a34f0-603f-437a-acaf-8f22d86e7bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:31.643992 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a34f0-603f-437a-acaf-8f22d86e7bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:31.645183 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a34f0-603f-437a-acaf-8f22d86e7bf5 None admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:31.646000 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a34f0-603f-437a-acaf-8f22d86e7bf5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:31.646503 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a34f0-603f-437a-acaf-8f22d86e7bf5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:31.648230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a34f0-603f-437a-acaf-8f22d86e7bf5 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:31.667089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 208/413] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:08:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:31.668079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a34f0-603f-437a-acaf-8f22d86e7bf5 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:31.679782 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:31.695630 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657123c-ffb4-49ca-921f-667b8dbb80cf None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:31.740378 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92dadd-c9a4-4f5f-9d74-5a7786be262f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:31.741414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92dadd-c9a4-4f5f-9d74-5a7786be262f None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:31.741706 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92dadd-c9a4-4f5f-9d74-5a7786be262f None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:31.741983 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92dadd-c9a4-4f5f-9d74-5a7786be262f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:31.751312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e92dadd-c9a4-4f5f-9d74-5a7786be262f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:31.752793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e92dadd-c9a4-4f5f-9d74-5a7786be262f None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:31.768219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-6e92dadd-c9a4-4f5f-9d74-5a7786be262f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:31.768219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:31.768219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:31.768219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:31.768219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:31.768219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:31.768219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:31.768219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:31.768219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:31.768219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:31.768219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:31.768219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:31.768219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:08:31.768219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:08:31.768219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:08:31.769625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:08:31.769625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:31.769625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:31.769625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:31.769625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:31.769625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:31.769625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:31.769625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:31.769625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:31.769625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:31.769625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:31.769625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:31.769625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:31.769625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:31.769625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:31.769625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:31.769625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:31.770454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:08:31.770454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:08:31.770454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:08:31.770454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:08:31.770454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:08:31.770454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:08:31.770454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:31.770454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:31.774887 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 206/414] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:31.775721 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92dadd-c9a4-4f5f-9d74-5a7786be262f None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:31.790986 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a34f0-603f-437a-acaf-8f22d86e7bf5 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:31.832550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c82c5-b45a-4029-835a-ade0caf8f7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:31.833456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c82c5-b45a-4029-835a-ade0caf8f7a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:31.833775 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c82c5-b45a-4029-835a-ade0caf8f7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:31.834504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c82c5-b45a-4029-835a-ade0caf8f7a2 None admin] PATH_INFO: `/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/d620cad0ef21453a9cab61f728913cdd/roles/b0f9a090148d4a0eb6456472edec3d5a` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:31.854831 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c82c5-b45a-4029-835a-ade0caf8f7a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b2c33250ef84d79a4a4742fb4252156, user_id=d620cad0ef21453a9cab61f728913cdd, role_id=b0f9a090148d4a0eb6456472edec3d5a)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:31.857187 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c82c5-b45a-4029-835a-ade0caf8f7a2 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:31.879886 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 209/415] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:08:31 2022] PUT /identity/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/d620cad0ef21453a9cab61f728913cdd/roles/b0f9a090148d4a0eb6456472edec3d5a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:08:31.880785 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c82c5-b45a-4029-835a-ade0caf8f7a2 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:31.915820 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92dadd-c9a4-4f5f-9d74-5a7786be262f None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:31.934624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:31.956378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:32.005097 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c82c5-b45a-4029-835a-ade0caf8f7a2 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:32.012952 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc1906-8d41-4197-9f22-138db87391cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:32.013625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc1906-8d41-4197-9f22-138db87391cf None admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:32.014170 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc1906-8d41-4197-9f22-138db87391cf None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:32.014446 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc1906-8d41-4197-9f22-138db87391cf None admin] PATH_INFO: `/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/5ce5ec441b06408bbadd6fc040af0da5/roles/b0f9a090148d4a0eb6456472edec3d5a` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:32.043696 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14567b1d-848a-4ef0-b517-3196041fd1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:32.044401 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14567b1d-848a-4ef0-b517-3196041fd1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:32.045230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14567b1d-848a-4ef0-b517-3196041fd1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:32.045502 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14567b1d-848a-4ef0-b517-3196041fd1a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:32.046198 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14567b1d-848a-4ef0-b517-3196041fd1a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:32.046500 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14567b1d-848a-4ef0-b517-3196041fd1a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:32.048134 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14567b1d-848a-4ef0-b517-3196041fd1a7 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:32.056027 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc1906-8d41-4197-9f22-138db87391cf None admin] RBAC: Authorizing `identity:create_grant(project_id=8b2c33250ef84d79a4a4742fb4252156, user_id=5ce5ec441b06408bbadd6fc040af0da5, role_id=b0f9a090148d4a0eb6456472edec3d5a)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:32.057487 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc1906-8d41-4197-9f22-138db87391cf None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:32.061606 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 210/416] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:08:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:32.068593 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14567b1d-848a-4ef0-b517-3196041fd1a7 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:32.094327 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 207/417] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:08:31 2022] PUT /identity/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/5ce5ec441b06408bbadd6fc040af0da5/roles/b0f9a090148d4a0eb6456472edec3d5a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:08:32.198585 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14567b1d-848a-4ef0-b517-3196041fd1a7 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:32.218303 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:32.241008 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ecab5-4c5a-4908-9495-94bf4b43643f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:32.241685 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ecab5-4c5a-4908-9495-94bf4b43643f None admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:32.241962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ecab5-4c5a-4908-9495-94bf4b43643f None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:32.242233 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ecab5-4c5a-4908-9495-94bf4b43643f None admin] PATH_INFO: `/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/d34dc4e7c5474b6b96977019328d8890/roles/b0f9a090148d4a0eb6456472edec3d5a` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:32.258803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ecab5-4c5a-4908-9495-94bf4b43643f None admin] RBAC: Authorizing `identity:create_grant(project_id=8b2c33250ef84d79a4a4742fb4252156, user_id=d34dc4e7c5474b6b96977019328d8890, role_id=b0f9a090148d4a0eb6456472edec3d5a)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:32.260304 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ecab5-4c5a-4908-9495-94bf4b43643f None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:32.277442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 211/418] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:08:32 2022] PUT /identity/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/d34dc4e7c5474b6b96977019328d8890/roles/b0f9a090148d4a0eb6456472edec3d5a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:08:32.884743 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38311425-03ec-4820-aa10-5f3e128b95c1 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:32.885137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38311425-03ec-4820-aa10-5f3e128b95c1 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:32.885335 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38311425-03ec-4820-aa10-5f3e128b95c1 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:32.886886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 208/419] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:32.892479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b5d9f-f777-41d4-8df5-d1c64f7efeab None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:32.892755 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b5d9f-f777-41d4-8df5-d1c64f7efeab None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:32.893026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b5d9f-f777-41d4-8df5-d1c64f7efeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:32.916674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-3f6b5d9f-f777-41d4-8df5-d1c64f7efeab None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:32.917888 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-3f6b5d9f-f777-41d4-8df5-d1c64f7efeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:32.978595 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796e03f-9ab2-447b-a06a-723bc763de21 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:32.979232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796e03f-9ab2-447b-a06a-723bc763de21 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:32.979744 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796e03f-9ab2-447b-a06a-723bc763de21 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:32.982659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 209/420] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:32.987299 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f22291-a048-4cc3-8944-426788379a08 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:32.987516 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f22291-a048-4cc3-8944-426788379a08 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:32.987880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f22291-a048-4cc3-8944-426788379a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:33.011309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 212/421] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:33.015448 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-49f22291-a048-4cc3-8944-426788379a08 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:33.016664 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-49f22291-a048-4cc3-8944-426788379a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:33.093638 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 210/422] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:33.442577 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b5d9f-f777-41d4-8df5-d1c64f7efeab None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:33.494214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f22291-a048-4cc3-8944-426788379a08 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:33.554045 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:33.560066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b5d9f-f777-41d4-8df5-d1c64f7efeab None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:33.599860 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc76d43-7c0c-44ee-859e-29350ad53496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:33.600763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc76d43-7c0c-44ee-859e-29350ad53496 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:33.601188 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc76d43-7c0c-44ee-859e-29350ad53496 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:33.601605 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc76d43-7c0c-44ee-859e-29350ad53496 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:33.617120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc76d43-7c0c-44ee-859e-29350ad53496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:33.628288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-0bc76d43-7c0c-44ee-859e-29350ad53496 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:33.628288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:33.628288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:33.628288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:33.628288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:33.628288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:33.628288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:33.628288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:33.628288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:33.628288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:33.628288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:33.628288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:33.628288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:33.628288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:33.628288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:33.628288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:33.629932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:33.629932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:33.629932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:33.629932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:33.629932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:33.629932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:33.629932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:33.629932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:33.629932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:33.629932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:33.629932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:33.629932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:33.629932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:33.629932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:33.629932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:33.629932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:33.630873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:33.630873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:33.630873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:33.630873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:33.630873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:33.630873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:33.630873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:33.630873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:33.630873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:33.630873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:33.630873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:33.630873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:33.635124 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 213/423] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Oct 21 21:08:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:33.640637 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc76d43-7c0c-44ee-859e-29350ad53496 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:33.671562 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:33.695447 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f22291-a048-4cc3-8944-426788379a08 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:33.744554 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab158c-0e2c-4320-aedc-b682797fa063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:33.745414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab158c-0e2c-4320-aedc-b682797fa063 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:33.745871 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab158c-0e2c-4320-aedc-b682797fa063 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:33.746312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab158c-0e2c-4320-aedc-b682797fa063 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:33.756963 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab158c-0e2c-4320-aedc-b682797fa063 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:33.762140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-9bab158c-0e2c-4320-aedc-b682797fa063 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:33.762140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:33.762140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:33.762140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:33.762140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:33.762140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:33.762140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:33.762140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:33.762140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:33.762140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:33.762140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:33.762140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:33.762140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:33.762140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:33.762140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:33.762140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:33.762994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:33.762994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:33.762994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:33.762994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:33.762994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:33.762994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:33.762994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:33.762994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:33.762994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:33.762994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:33.762994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:33.762994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:33.762994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:33.762994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:33.762994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:33.762994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:33.764098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:33.764098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:33.764098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:33.764098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:33.764098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:33.764098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:33.764098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:33.764098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:33.764098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:33.764098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:33.764098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:33.764098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:33.766873 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 211/424] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Oct 21 21:08:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:33.773827 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab158c-0e2c-4320-aedc-b682797fa063 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:33.813981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc76d43-7c0c-44ee-859e-29350ad53496 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:33.852908 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34213188-91aa-4ede-903c-2780c5db81a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:33.853521 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34213188-91aa-4ede-903c-2780c5db81a5 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:33.853796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34213188-91aa-4ede-903c-2780c5db81a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:33.854064 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34213188-91aa-4ede-903c-2780c5db81a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:33.854723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34213188-91aa-4ede-903c-2780c5db81a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:33.855022 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34213188-91aa-4ede-903c-2780c5db81a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:33.856763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34213188-91aa-4ede-903c-2780c5db81a5 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:33.878047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 214/425] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Oct 21 21:08:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:33.883281 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34213188-91aa-4ede-903c-2780c5db81a5 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:33.907332 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab158c-0e2c-4320-aedc-b682797fa063 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:33.933253 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:33.944032 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee5235-9006-4c33-914b-6556f5b0c8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:33.944913 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee5235-9006-4c33-914b-6556f5b0c8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:33.945433 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee5235-9006-4c33-914b-6556f5b0c8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:33.945955 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee5235-9006-4c33-914b-6556f5b0c8f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:33.946885 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee5235-9006-4c33-914b-6556f5b0c8f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:33.947401 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee5235-9006-4c33-914b-6556f5b0c8f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:33.949111 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee5235-9006-4c33-914b-6556f5b0c8f8 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:33.958583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 212/426] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Oct 21 21:08:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:33.963748 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee5235-9006-4c33-914b-6556f5b0c8f8 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:33.980636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:34.072959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee5235-9006-4c33-914b-6556f5b0c8f8 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:34.135676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a07a98-8710-4d3b-9792-c5774cdc2abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:34.136307 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a07a98-8710-4d3b-9792-c5774cdc2abf None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:34.136583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a07a98-8710-4d3b-9792-c5774cdc2abf None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:34.136850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a07a98-8710-4d3b-9792-c5774cdc2abf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:34.153275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34213188-91aa-4ede-903c-2780c5db81a5 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:34.155288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a07a98-8710-4d3b-9792-c5774cdc2abf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:34.156746 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a07a98-8710-4d3b-9792-c5774cdc2abf None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:34.181268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-88a07a98-8710-4d3b-9792-c5774cdc2abf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:34.181268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:34.181268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:34.181268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:34.181268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:34.181268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:34.181268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:34.181268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:34.181268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:34.181268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:34.181268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:34.181268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:34.181268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:34.181268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:34.181268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:34.181268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:34.182213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:34.182213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:34.182213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:34.182213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:34.182213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:34.182213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:34.182213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:34.182213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:34.182213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:34.182213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:34.182213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:34.182213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:34.182213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:34.182213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:34.182213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:34.182213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:34.182213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:34.183678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:34.183678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:34.183678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:34.183678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 213/427] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:34.188788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a07a98-8710-4d3b-9792-c5774cdc2abf None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:34.207102 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff48623-2205-40c0-959c-50cf3d8d536e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:34.207695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff48623-2205-40c0-959c-50cf3d8d536e None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:34.207977 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff48623-2205-40c0-959c-50cf3d8d536e None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:34.208241 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff48623-2205-40c0-959c-50cf3d8d536e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:34.227179 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff48623-2205-40c0-959c-50cf3d8d536e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:34.228995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff48623-2205-40c0-959c-50cf3d8d536e None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:34.263592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-9ff48623-2205-40c0-959c-50cf3d8d536e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:34.263592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:34.263592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:34.263592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:34.263592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:34.263592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:34.263592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:34.263592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:34.263592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:34.263592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:34.263592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:34.263592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:34.263592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:34.263592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:34.263592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:34.263592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:34.264793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:34.264793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:34.264793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:34.264793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:34.264793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:34.264793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:34.264793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:34.264793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:34.264793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:34.264793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:34.264793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:34.264793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:34.264793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:34.264793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:34.264793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:34.264793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:34.264793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:34.265624 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:34.265624 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:34.265624 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:34.265783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:34.267926 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 215/428] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:08:34.273472 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff48623-2205-40c0-959c-50cf3d8d536e None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:34.302921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a07a98-8710-4d3b-9792-c5774cdc2abf None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:34.342023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cf3dd-3767-47f9-b211-4e9728c1559d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:34.342023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cf3dd-3767-47f9-b211-4e9728c1559d None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:34.342023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cf3dd-3767-47f9-b211-4e9728c1559d None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:34.342023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cf3dd-3767-47f9-b211-4e9728c1559d None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:34.342423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13cf3dd-3767-47f9-b211-4e9728c1559d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:34.342423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13cf3dd-3767-47f9-b211-4e9728c1559d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:34.342423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13cf3dd-3767-47f9-b211-4e9728c1559d None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:34.354068 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 214/429] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:34.359829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cf3dd-3767-47f9-b211-4e9728c1559d None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:34.412683 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:34.470568 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff48623-2205-40c0-959c-50cf3d8d536e None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:34.530408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e5027-3a93-4dee-a88e-83ed4a4238ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:34.531168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e5027-3a93-4dee-a88e-83ed4a4238ba None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:34.531447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e5027-3a93-4dee-a88e-83ed4a4238ba None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:34.532052 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e5027-3a93-4dee-a88e-83ed4a4238ba None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:34.532959 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e5027-3a93-4dee-a88e-83ed4a4238ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:34.533259 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e5027-3a93-4dee-a88e-83ed4a4238ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:34.535166 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e5027-3a93-4dee-a88e-83ed4a4238ba None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:34.541127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cf3dd-3767-47f9-b211-4e9728c1559d None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:34.549658 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 216/430] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:34.551975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc5991-6bfd-41b7-8100-612c352989e4 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:34.552382 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc5991-6bfd-41b7-8100-612c352989e4 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:34.553022 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc5991-6bfd-41b7-8100-612c352989e4 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:34.554555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 217/431] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:34.555390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc5991-6bfd-41b7-8100-612c352989e4 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:34.573408 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:34.590684 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a922f2a-a035-418b-9305-4b16d05f224d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:34.591608 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a922f2a-a035-418b-9305-4b16d05f224d None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:34.591891 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a922f2a-a035-418b-9305-4b16d05f224d None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:34.592409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a922f2a-a035-418b-9305-4b16d05f224d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:34.601984 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a922f2a-a035-418b-9305-4b16d05f224d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:34.603597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a922f2a-a035-418b-9305-4b16d05f224d None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:34.620238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-3a922f2a-a035-418b-9305-4b16d05f224d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 21 21:08:34.620238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:34.620238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:34.620238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:34.620238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:34.620238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:34.620238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:34.620238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:34.620238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:34.620238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:34.620238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:34.620238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:34.620238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:08:34.620238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:08:34.620238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:08:34.620238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:34.621147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:34.622645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:34.622645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:34.622645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:08:34.622645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:08:34.622645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:08:34.622645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:08:34.622645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:08:34.622645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:08:34.622645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 21 21:08:34.622645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:34.622645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 215/432] 10.222.0.13 () {56 vars in 1093 bytes} [Fri Oct 21 21:08:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:08:34.624057 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c278a-5c21-4ff6-9f6c-017eb550c091 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:34.624057 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c278a-5c21-4ff6-9f6c-017eb550c091 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:34.625819 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c278a-5c21-4ff6-9f6c-017eb550c091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:34.652609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-1e7c278a-5c21-4ff6-9f6c-017eb550c091 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:34.654671 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1e7c278a-5c21-4ff6-9f6c-017eb550c091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:34.678699 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc5991-6bfd-41b7-8100-612c352989e4 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:34.712005 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:34.717952 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67696e3-2402-4a0a-a80b-06047620a813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:34.718630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67696e3-2402-4a0a-a80b-06047620a813 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:34.718914 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67696e3-2402-4a0a-a80b-06047620a813 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:34.719185 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67696e3-2402-4a0a-a80b-06047620a813 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:34.727291 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67696e3-2402-4a0a-a80b-06047620a813 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:34.728773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67696e3-2402-4a0a-a80b-06047620a813 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:34.739218 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 216/433] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:34.740299 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c278a-5c21-4ff6-9f6c-017eb550c091 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:34.750805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-d67696e3-2402-4a0a-a80b-06047620a813 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 21 21:08:34.750805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:34.750805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:34.750805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:34.750805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:34.750805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:34.750805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:34.750805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:34.750805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:34.750805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:34.750805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:34.750805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:34.750805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:08:34.750805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:08:34.750805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:08:34.750805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:34.751909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:34.752786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:34.752786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:34.752786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:08:34.752786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:08:34.752786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:08:34.752786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:08:34.752786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:08:34.752786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:08:34.752786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 21 21:08:34.752786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:34.752786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 218/434] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:08:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:34.757955 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67696e3-2402-4a0a-a80b-06047620a813 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:34.880979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c278a-5c21-4ff6-9f6c-017eb550c091 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:34.899684 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67696e3-2402-4a0a-a80b-06047620a813 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:34.904750 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:34.944737 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfce18-d44d-4b9e-bfc3-9243ea5bfd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:34.945719 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfce18-d44d-4b9e-bfc3-9243ea5bfd3d None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:34.946003 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfce18-d44d-4b9e-bfc3-9243ea5bfd3d None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:34.946275 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfce18-d44d-4b9e-bfc3-9243ea5bfd3d None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:34.947964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfce18-d44d-4b9e-bfc3-9243ea5bfd3d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:34.948579 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfce18-d44d-4b9e-bfc3-9243ea5bfd3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:34.951387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfce18-d44d-4b9e-bfc3-9243ea5bfd3d None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:34.955386 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c50550-db69-4240-9cde-e6ffc63fe41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:34.956047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c50550-db69-4240-9cde-e6ffc63fe41a None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:34.956328 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c50550-db69-4240-9cde-e6ffc63fe41a None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:34.956629 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c50550-db69-4240-9cde-e6ffc63fe41a None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:34.957372 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c50550-db69-4240-9cde-e6ffc63fe41a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:34.958446 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c50550-db69-4240-9cde-e6ffc63fe41a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:34.959135 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c50550-db69-4240-9cde-e6ffc63fe41a None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:34.980235 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 219/435] 10.222.0.13 () {56 vars in 1135 bytes} [Fri Oct 21 21:08:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:34.981651 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85042c99-662a-4784-bff3-d4723bae07dc None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:34.981940 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85042c99-662a-4784-bff3-d4723bae07dc None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:34.983061 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85042c99-662a-4784-bff3-d4723bae07dc None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:34.983445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 220/436] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:34.989150 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e085d-9520-440c-bb40-ea5c0eee727d None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:34.989565 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e085d-9520-440c-bb40-ea5c0eee727d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:34.989937 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e085d-9520-440c-bb40-ea5c0eee727d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:35.009774 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 217/437] 10.222.0.13 () {56 vars in 1133 bytes} [Fri Oct 21 21:08:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:35.011024 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfce18-d44d-4b9e-bfc3-9243ea5bfd3d None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:35.011984 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-116e085d-9520-440c-bb40-ea5c0eee727d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:35.013283 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-116e085d-9520-440c-bb40-ea5c0eee727d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:35.072937 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:35.097829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 221/438] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:35.098658 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e085d-9520-440c-bb40-ea5c0eee727d None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:35.132711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfce18-d44d-4b9e-bfc3-9243ea5bfd3d None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:35.168875 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959d7af-652b-49e8-a719-0d2c94a91890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:35.169511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d7af-652b-49e8-a719-0d2c94a91890 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:35.169790 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d7af-652b-49e8-a719-0d2c94a91890 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:35.170062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d7af-652b-49e8-a719-0d2c94a91890 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:35.182848 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959d7af-652b-49e8-a719-0d2c94a91890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:35.184674 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959d7af-652b-49e8-a719-0d2c94a91890 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:35.199953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-d959d7af-652b-49e8-a719-0d2c94a91890 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:35.199953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:35.199953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:35.199953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:35.199953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:35.199953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:35.199953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:35.199953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:35.199953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:35.199953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:35.199953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:35.199953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:35.199953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:35.199953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:35.199953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:35.199953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:35.200910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:35.200910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:35.200910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:35.200910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:35.200910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:35.200910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:35.200910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:35.200910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:35.200910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:35.200910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:35.200910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:35.200910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:35.200910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:35.200910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:35.200910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:35.200910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:35.200910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:35.201744 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:35.201744 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:35.201744 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:35.201744 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 218/439] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:35.202522 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ddd3d-0a0c-433e-8249-e0d2d7b9d84c None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:35.202901 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ddd3d-0a0c-433e-8249-e0d2d7b9d84c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:35.203055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ddd3d-0a0c-433e-8249-e0d2d7b9d84c None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:35.207648 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 219/440] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:35.207648 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ddd3d-0a0c-433e-8249-e0d2d7b9d84c None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:35.214087 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e085d-9520-440c-bb40-ea5c0eee727d None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:35.233254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:35.258493 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde018ae-5bbc-4889-8ac1-ad4dc8cfe312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:35.259135 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde018ae-5bbc-4889-8ac1-ad4dc8cfe312 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:35.259408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde018ae-5bbc-4889-8ac1-ad4dc8cfe312 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:35.259710 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde018ae-5bbc-4889-8ac1-ad4dc8cfe312 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:35.308495 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:35.318731 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ddd3d-0a0c-433e-8249-e0d2d7b9d84c None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:35.328539 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde018ae-5bbc-4889-8ac1-ad4dc8cfe312 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:35.330013 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde018ae-5bbc-4889-8ac1-ad4dc8cfe312 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:35.354283 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8dab0-0b03-40e8-a9d1-6f9b2d1866aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:35.355177 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8dab0-0b03-40e8-a9d1-6f9b2d1866aa None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:35.355663 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8dab0-0b03-40e8-a9d1-6f9b2d1866aa None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:35.356159 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8dab0-0b03-40e8-a9d1-6f9b2d1866aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:35.367984 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8dab0-0b03-40e8-a9d1-6f9b2d1866aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:35.373800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-d2b8dab0-0b03-40e8-a9d1-6f9b2d1866aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:35.373800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:35.373800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:35.373800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:35.373800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:35.373800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:35.373800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:35.373800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:35.373800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:35.373800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:35.373800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:35.373800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:35.373800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:35.373800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:35.373800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:35.373800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:35.374759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:35.374759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:35.374759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:35.374759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:35.374759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:35.374759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:35.374759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:35.374759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:35.374759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:35.374759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:35.374759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:35.374759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:35.374759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:35.374759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:35.374759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:35.374759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:35.375632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:35.375632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:35.375632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:35.375632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:35.375632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:35.375632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:35.375632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:35.375632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:35.375632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:35.375632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:35.375632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:35.375632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:35.377504 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 220/441] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Oct 21 21:08:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:35.378999 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8dab0-0b03-40e8-a9d1-6f9b2d1866aa None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:35.438901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 222/442] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:08:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:08:35.440183 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd468a5d-9cca-4a47-8c59-0da1eac7d01e None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:35.440623 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd468a5d-9cca-4a47-8c59-0da1eac7d01e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:35.441629 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd468a5d-9cca-4a47-8c59-0da1eac7d01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:35.461325 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-dd468a5d-9cca-4a47-8c59-0da1eac7d01e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:35.462509 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-dd468a5d-9cca-4a47-8c59-0da1eac7d01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:35.464225 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:35.482607 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8dab0-0b03-40e8-a9d1-6f9b2d1866aa None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:35.517006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dff4cd-c7f8-4b24-91a5-221369b2ae4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:35.517533 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dff4cd-c7f8-4b24-91a5-221369b2ae4e None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:35.517809 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dff4cd-c7f8-4b24-91a5-221369b2ae4e None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:35.518076 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dff4cd-c7f8-4b24-91a5-221369b2ae4e None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:35.518710 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dff4cd-c7f8-4b24-91a5-221369b2ae4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:35.519040 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dff4cd-c7f8-4b24-91a5-221369b2ae4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:35.520438 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dff4cd-c7f8-4b24-91a5-221369b2ae4e None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:35.531977 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 221/443] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:35.532777 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dff4cd-c7f8-4b24-91a5-221369b2ae4e None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:35.536570 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 223/444] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:35.537200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd468a5d-9cca-4a47-8c59-0da1eac7d01e None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:35.549605 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:35.645229 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd468a5d-9cca-4a47-8c59-0da1eac7d01e None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:35.648407 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dff4cd-c7f8-4b24-91a5-221369b2ae4e None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:35.684405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a863d-31bb-45f0-88ee-65d98c94354c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:35.684580 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c9674-b4ab-49ec-b531-deb2426b7991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:35.685008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a863d-31bb-45f0-88ee-65d98c94354c None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:35.685283 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a863d-31bb-45f0-88ee-65d98c94354c None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:35.685379 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c9674-b4ab-49ec-b531-deb2426b7991 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:35.685545 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a863d-31bb-45f0-88ee-65d98c94354c None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:35.685647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c9674-b4ab-49ec-b531-deb2426b7991 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:35.685874 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c9674-b4ab-49ec-b531-deb2426b7991 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:35.686198 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a863d-31bb-45f0-88ee-65d98c94354c None admin] RBAC: Adding query filter params (name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:35.686497 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a863d-31bb-45f0-88ee-65d98c94354c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:35.687869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a863d-31bb-45f0-88ee-65d98c94354c None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:35.696575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 222/445] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Oct 21 21:08:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:35.697344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a863d-31bb-45f0-88ee-65d98c94354c None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:35.701402 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:35.702883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c9674-b4ab-49ec-b531-deb2426b7991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:35.708953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-bb7c9674-b4ab-49ec-b531-deb2426b7991 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:35.708953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:35.708953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:35.708953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:35.708953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:35.708953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:35.708953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:35.708953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:35.708953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:35.708953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:35.708953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:35.708953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:35.708953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:35.708953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:35.708953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:35.708953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:35.709818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:35.709818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:35.709818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:35.709818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:35.709818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:35.709818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:35.709818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:35.709818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:35.709818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:35.709818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:35.709818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:35.709818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:35.709818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:35.709818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:35.709818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:35.709818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:35.710618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:35.710618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:35.710618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:35.710618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:35.710618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:35.710618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:35.710618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:35.710618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:35.710618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:35.710618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:35.710618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:35.710618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:35.711863 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 224/446] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Oct 21 21:08:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:35.711863 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c9674-b4ab-49ec-b531-deb2426b7991 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:35.789895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:35.811907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a863d-31bb-45f0-88ee-65d98c94354c None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:35.821718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c9674-b4ab-49ec-b531-deb2426b7991 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:35.843766 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315fc34-5d86-44aa-a718-64fdee392c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:35.844401 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315fc34-5d86-44aa-a718-64fdee392c5b None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:35.844697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315fc34-5d86-44aa-a718-64fdee392c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:35.844966 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315fc34-5d86-44aa-a718-64fdee392c5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:35.853744 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13197d64-af29-4253-8ef9-1dc3954eb996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:35.854359 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13197d64-af29-4253-8ef9-1dc3954eb996 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:35.854637 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13197d64-af29-4253-8ef9-1dc3954eb996 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:35.854906 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13197d64-af29-4253-8ef9-1dc3954eb996 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:35.855529 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5315fc34-5d86-44aa-a718-64fdee392c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:35.856979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5315fc34-5d86-44aa-a718-64fdee392c5b None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:35.861380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13197d64-af29-4253-8ef9-1dc3954eb996 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:35.863397 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13197d64-af29-4253-8ef9-1dc3954eb996 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:35.871298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-5315fc34-5d86-44aa-a718-64fdee392c5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:35.871298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:35.871298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:35.871298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:35.871298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:35.871298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:35.871298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:35.871298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:35.871298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:35.871298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:35.871298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:35.871298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:35.871298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:35.871298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:35.871298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:35.871298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:35.872237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:35.872237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:35.872237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:35.872237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:35.872237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:35.872237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:35.872237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:35.872237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:35.872237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:35.872237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:35.872237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:35.872237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:35.872237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:35.872237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:35.872237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:35.872237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:35.872237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:35.873341 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:35.873341 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:35.873341 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:35.873341 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 223/447] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:35.873702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315fc34-5d86-44aa-a718-64fdee392c5b None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:35.874299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-13197d64-af29-4253-8ef9-1dc3954eb996 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:35.874299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:35.874299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:35.874299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:35.874299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:35.874299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:35.874299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:35.874299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:35.874299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:35.874299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:35.874299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:35.874299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:35.874299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:08:35.874299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:08:35.874299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:08:35.875231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:08:35.875231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:35.875231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:35.875231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:35.875231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:35.875231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:35.875231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:35.875231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:35.875231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:35.875231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:35.875231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:35.875231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:35.875231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:35.875231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:35.875231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:35.875231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:35.875231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:35.876154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:08:35.876154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:08:35.876154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:08:35.876154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:08:35.876154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:08:35.876154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:08:35.876154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:35.876154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:35.876154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 225/448] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:35.876154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13197d64-af29-4253-8ef9-1dc3954eb996 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:35.931713 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:35.974722 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13197d64-af29-4253-8ef9-1dc3954eb996 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:35.975202 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315fc34-5d86-44aa-a718-64fdee392c5b None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:36.001747 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:36.007681 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d323f-644d-40ef-a0a6-6adda6a02fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:36.007920 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cda038-86c9-43ab-9e3a-6c6819287bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:36.008308 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d323f-644d-40ef-a0a6-6adda6a02fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:36.008521 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cda038-86c9-43ab-9e3a-6c6819287bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:36.008629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d323f-644d-40ef-a0a6-6adda6a02fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:36.008806 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cda038-86c9-43ab-9e3a-6c6819287bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:36.008894 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d323f-644d-40ef-a0a6-6adda6a02fc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:36.009078 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cda038-86c9-43ab-9e3a-6c6819287bc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:36.009733 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cda038-86c9-43ab-9e3a-6c6819287bc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:36.010039 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cda038-86c9-43ab-9e3a-6c6819287bc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:36.011427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cda038-86c9-43ab-9e3a-6c6819287bc7 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:36.019846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d323f-644d-40ef-a0a6-6adda6a02fc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:36.021273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d323f-644d-40ef-a0a6-6adda6a02fc7 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:36.022515 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 226/449] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Oct 21 21:08:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:36.027194 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cda038-86c9-43ab-9e3a-6c6819287bc7 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:36.039802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-7f7d323f-644d-40ef-a0a6-6adda6a02fc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:36.039802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:36.039802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:36.039802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:36.039802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:36.039802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:36.039802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:36.039802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:36.039802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:36.039802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:36.039802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:36.039802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:36.039802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:36.039802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:36.039802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:36.039802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:36.040714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:36.040714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:36.040714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:36.040714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:36.040714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:36.040714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:36.040714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:36.040714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:36.040714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:36.040714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:36.040714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:36.040714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:36.040714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:36.040714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:36.040714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:36.040714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:36.040714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:36.041552 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:36.041552 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:36.041552 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:36.041552 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 224/450] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:36.041909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d323f-644d-40ef-a0a6-6adda6a02fc7 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:36.122875 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cda038-86c9-43ab-9e3a-6c6819287bc7 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:36.138447 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d323f-644d-40ef-a0a6-6adda6a02fc7 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:36.147697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:36.156152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71154bf3-44ab-488f-9cd1-17538d3a4e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:36.156768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71154bf3-44ab-488f-9cd1-17538d3a4e95 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:36.157044 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71154bf3-44ab-488f-9cd1-17538d3a4e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:36.157312 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71154bf3-44ab-488f-9cd1-17538d3a4e95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:36.167417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71154bf3-44ab-488f-9cd1-17538d3a4e95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:36.171927 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400d74b-2b71-49d0-86ad-e930348a556e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:36.172516 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d74b-2b71-49d0-86ad-e930348a556e None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:36.172628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-71154bf3-44ab-488f-9cd1-17538d3a4e95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:36.172628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:36.172628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:36.172628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:36.172628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:36.172628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:36.172628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:36.172628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:36.172628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:36.172628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:36.172628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:36.172628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:36.172628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:36.172628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:36.172628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:36.172628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:36.173625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:36.173625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:36.173625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:36.173625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:36.173625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:36.173625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:36.173625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:36.173625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:36.173625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:36.173625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:36.173625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:36.173625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:36.173625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:36.173625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:36.173625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:36.173625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:36.174410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:36.174410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:36.174410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:36.174410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:36.174410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:36.174410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:36.174410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:36.174410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:36.174410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:36.174410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:36.174410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:36.174410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:36.175519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d74b-2b71-49d0-86ad-e930348a556e None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:36.175519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d74b-2b71-49d0-86ad-e930348a556e None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:36.175519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400d74b-2b71-49d0-86ad-e930348a556e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:36.175751 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 227/451] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Oct 21 21:08:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:36.175817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400d74b-2b71-49d0-86ad-e930348a556e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:36.175881 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71154bf3-44ab-488f-9cd1-17538d3a4e95 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:36.175945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400d74b-2b71-49d0-86ad-e930348a556e None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:36.189361 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 225/452] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:36.190449 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400d74b-2b71-49d0-86ad-e930348a556e None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:36.227404 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:36.273440 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71154bf3-44ab-488f-9cd1-17538d3a4e95 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:36.288835 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400d74b-2b71-49d0-86ad-e930348a556e None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:36.306042 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a527f-9bf3-4d01-89f3-1bc0829e5563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:36.306582 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a527f-9bf3-4d01-89f3-1bc0829e5563 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:36.306865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a527f-9bf3-4d01-89f3-1bc0829e5563 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:36.307142 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a527f-9bf3-4d01-89f3-1bc0829e5563 None admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:36.307849 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a527f-9bf3-4d01-89f3-1bc0829e5563 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:36.308584 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a527f-9bf3-4d01-89f3-1bc0829e5563 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:36.310029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a527f-9bf3-4d01-89f3-1bc0829e5563 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:36.321548 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 228/453] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:08:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:36.322323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a527f-9bf3-4d01-89f3-1bc0829e5563 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:36.327000 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a6fc9-12fc-4462-bcb2-2fad3dcc6339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:36.327730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a6fc9-12fc-4462-bcb2-2fad3dcc6339 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:36.328009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a6fc9-12fc-4462-bcb2-2fad3dcc6339 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:36.328279 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a6fc9-12fc-4462-bcb2-2fad3dcc6339 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:36.338541 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a6fc9-12fc-4462-bcb2-2fad3dcc6339 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:36.339996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a6fc9-12fc-4462-bcb2-2fad3dcc6339 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:36.354636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-eb2a6fc9-12fc-4462-bcb2-2fad3dcc6339 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:36.354636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:36.354636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:36.354636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:36.354636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:36.354636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:36.354636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:36.354636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:36.354636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:36.354636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:36.354636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:36.354636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:36.354636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:36.354636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:36.354636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:36.354636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:36.355926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:36.355926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:36.355926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:36.355926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:36.355926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:36.355926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:36.355926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:36.355926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:36.355926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:36.355926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:36.355926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:36.355926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:36.355926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:36.355926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:36.355926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:36.355926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:36.355926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:36.357144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:36.357144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:36.357144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:36.357144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 226/454] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:36.357144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a6fc9-12fc-4462-bcb2-2fad3dcc6339 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:36.382054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:36.429155 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a527f-9bf3-4d01-89f3-1bc0829e5563 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:36.442360 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:36.459480 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a6fc9-12fc-4462-bcb2-2fad3dcc6339 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:36.462504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa50637-b66d-43dd-85ec-bfa3fe06ab98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:36.463102 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa50637-b66d-43dd-85ec-bfa3fe06ab98 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:36.463375 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa50637-b66d-43dd-85ec-bfa3fe06ab98 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:36.463670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa50637-b66d-43dd-85ec-bfa3fe06ab98 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:36.464295 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa50637-b66d-43dd-85ec-bfa3fe06ab98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:36.464589 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa50637-b66d-43dd-85ec-bfa3fe06ab98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:36.465946 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa50637-b66d-43dd-85ec-bfa3fe06ab98 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:36.477274 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 229/455] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:36.477882 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa50637-b66d-43dd-85ec-bfa3fe06ab98 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:36.492208 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b25f38-6942-4503-8d83-0872c97c5fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:36.492810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b25f38-6942-4503-8d83-0872c97c5fdc None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:36.493091 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b25f38-6942-4503-8d83-0872c97c5fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:36.493368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b25f38-6942-4503-8d83-0872c97c5fdc None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:36.494024 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b25f38-6942-4503-8d83-0872c97c5fdc None admin] RBAC: Adding query filter params (name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:36.494327 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b25f38-6942-4503-8d83-0872c97c5fdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:36.495726 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b25f38-6942-4503-8d83-0872c97c5fdc None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:36.503892 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 227/456] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Oct 21 21:08:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:36.504643 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b25f38-6942-4503-8d83-0872c97c5fdc None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:36.578418 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa50637-b66d-43dd-85ec-bfa3fe06ab98 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:36.584225 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:36.603128 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b25f38-6942-4503-8d83-0872c97c5fdc None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:36.610683 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0288f9-525b-486e-be40-b23644c74367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:36.611302 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0288f9-525b-486e-be40-b23644c74367 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:36.611593 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0288f9-525b-486e-be40-b23644c74367 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:36.611872 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0288f9-525b-486e-be40-b23644c74367 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:36.618749 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0288f9-525b-486e-be40-b23644c74367 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:36.620215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0288f9-525b-486e-be40-b23644c74367 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:36.631106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-7d0288f9-525b-486e-be40-b23644c74367 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:36.631106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:36.631106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:36.631106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:36.631106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:36.631106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:36.631106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:36.631106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:36.631106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:36.631106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:36.631106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:36.631106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:36.631106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:08:36.631106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:08:36.631106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:08:36.632136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:08:36.632136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:36.632136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:36.632136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:36.632136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:36.632136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:36.632136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:36.632136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:36.632136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:36.632136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:36.632136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:36.632136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:36.632136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:36.632136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:36.632136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:36.632136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:36.632136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:36.633151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:08:36.633151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:08:36.633151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:08:36.633151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:08:36.633151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:08:36.633151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:08:36.633151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:36.633151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:36.633151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 230/457] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:36.636174 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03afc0a-f481-4028-86b9-b67deb35366d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:36.636371 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0288f9-525b-486e-be40-b23644c74367 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:36.636829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03afc0a-f481-4028-86b9-b67deb35366d None admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:36.637110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03afc0a-f481-4028-86b9-b67deb35366d None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:36.637387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03afc0a-f481-4028-86b9-b67deb35366d None admin] PATH_INFO: `/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/7ce720c8df014f9a8af062138aecd1b5/roles/b0f9a090148d4a0eb6456472edec3d5a` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:36.653173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03afc0a-f481-4028-86b9-b67deb35366d None admin] RBAC: Authorizing `identity:create_grant(project_id=8b2c33250ef84d79a4a4742fb4252156, user_id=7ce720c8df014f9a8af062138aecd1b5, role_id=b0f9a090148d4a0eb6456472edec3d5a)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:36.654689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03afc0a-f481-4028-86b9-b67deb35366d None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:36.667414 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:36.667537 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 228/458] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:08:36 2022] PUT /identity/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/7ce720c8df014f9a8af062138aecd1b5/roles/b0f9a090148d4a0eb6456472edec3d5a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:08:36.672572 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03afc0a-f481-4028-86b9-b67deb35366d None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:36.749389 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0288f9-525b-486e-be40-b23644c74367 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:36.787389 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e8352-8e3f-4704-a164-7ab0f875b1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:36.788052 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e8352-8e3f-4704-a164-7ab0f875b1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:36.788332 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e8352-8e3f-4704-a164-7ab0f875b1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:36.788609 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e8352-8e3f-4704-a164-7ab0f875b1f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:36.789261 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e8352-8e3f-4704-a164-7ab0f875b1f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:36.789567 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e8352-8e3f-4704-a164-7ab0f875b1f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:36.790984 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e8352-8e3f-4704-a164-7ab0f875b1f4 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:36.795029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03afc0a-f481-4028-86b9-b67deb35366d None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:36.804576 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 231/459] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:36.805344 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e8352-8e3f-4704-a164-7ab0f875b1f4 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:36.832660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:36.840506 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb121e-fd9e-447c-88fa-fd2e93d237c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:36.841943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb121e-fd9e-447c-88fa-fd2e93d237c4 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:36.841943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb121e-fd9e-447c-88fa-fd2e93d237c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:36.842209 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb121e-fd9e-447c-88fa-fd2e93d237c4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:36.851380 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb121e-fd9e-447c-88fa-fd2e93d237c4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:36.852841 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb121e-fd9e-447c-88fa-fd2e93d237c4 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:36.865230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-caeb121e-fd9e-447c-88fa-fd2e93d237c4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 21 21:08:36.865230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:36.865230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:36.865230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:36.865230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:36.865230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:36.865230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:36.865230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:36.865230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:36.865230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:36.865230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:36.865230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:36.865230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:08:36.865230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:08:36.865230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:08:36.865230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:36.866137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:36.867058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:36.867058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:36.867058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:08:36.867058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:08:36.867058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:08:36.867058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:08:36.867058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:08:36.867058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:08:36.867058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 21 21:08:36.867058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:36.867058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 229/460] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:36.867664 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb121e-fd9e-447c-88fa-fd2e93d237c4 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:36.906993 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:36.917712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e8352-8e3f-4704-a164-7ab0f875b1f4 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:36.950659 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80599f-4d8a-4cb8-94f7-412d88669506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:36.951285 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80599f-4d8a-4cb8-94f7-412d88669506 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:36.951562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80599f-4d8a-4cb8-94f7-412d88669506 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:36.951856 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80599f-4d8a-4cb8-94f7-412d88669506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:36.962501 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80599f-4d8a-4cb8-94f7-412d88669506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:36.963988 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80599f-4d8a-4cb8-94f7-412d88669506 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:36.978550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-3d80599f-4d8a-4cb8-94f7-412d88669506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:36.978550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:36.978550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:36.978550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:36.978550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:36.978550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:36.978550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:36.978550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:36.978550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:36.978550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:36.978550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:36.978550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:36.978550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:36.978550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:36.978550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:36.978550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:36.979452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:36.979452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:36.979452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:36.979452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:36.979452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:36.979452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:36.979452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:36.979452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:36.979452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:36.979452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:36.979452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:36.979452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:36.979452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:36.979452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:36.979452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:36.979452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:36.979452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:36.980318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:36.980318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:36.980318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:36.980318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 232/461] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:36.980806 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80599f-4d8a-4cb8-94f7-412d88669506 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:36.982614 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb121e-fd9e-447c-88fa-fd2e93d237c4 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:37.020047 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89756a09-4326-403d-b9a9-4a1c6372b484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:37.020747 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89756a09-4326-403d-b9a9-4a1c6372b484 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:37.021028 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89756a09-4326-403d-b9a9-4a1c6372b484 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:37.021298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89756a09-4326-403d-b9a9-4a1c6372b484 None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:37.021974 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89756a09-4326-403d-b9a9-4a1c6372b484 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:37.022276 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89756a09-4326-403d-b9a9-4a1c6372b484 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:37.023714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89756a09-4326-403d-b9a9-4a1c6372b484 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:37.036194 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 230/462] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:08:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:37.041480 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89756a09-4326-403d-b9a9-4a1c6372b484 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:37.087636 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80599f-4d8a-4cb8-94f7-412d88669506 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:37.096595 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:37.124331 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087cfb2-b8ec-491b-a9a7-81d4a92c2d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:37.125099 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087cfb2-b8ec-491b-a9a7-81d4a92c2d39 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:37.125438 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087cfb2-b8ec-491b-a9a7-81d4a92c2d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:37.125773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087cfb2-b8ec-491b-a9a7-81d4a92c2d39 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:37.133977 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087cfb2-b8ec-491b-a9a7-81d4a92c2d39 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:37.135562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087cfb2-b8ec-491b-a9a7-81d4a92c2d39 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:37.148449 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89756a09-4326-403d-b9a9-4a1c6372b484 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:37.149674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-e087cfb2-b8ec-491b-a9a7-81d4a92c2d39 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 21 21:08:37.149674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:37.149674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:37.149674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:37.149674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:37.149674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:37.149674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:37.149674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:37.149674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:37.149674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:37.149674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:37.149674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:37.149674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:08:37.149674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:08:37.149674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:08:37.149674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:37.150689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:37.151550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:37.151550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:37.151550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:08:37.151550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:08:37.151550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:08:37.151550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:08:37.151550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:08:37.151550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:08:37.151550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 21 21:08:37.151550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:37.151550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 233/463] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:37.152730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087cfb2-b8ec-491b-a9a7-81d4a92c2d39 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:37.157933 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:37.191859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3f4c8-0750-4efa-b08a-65cb7d205654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:37.192090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3f4c8-0750-4efa-b08a-65cb7d205654 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:37.192310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3f4c8-0750-4efa-b08a-65cb7d205654 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:37.192556 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3f4c8-0750-4efa-b08a-65cb7d205654 None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:37.193274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3f4c8-0750-4efa-b08a-65cb7d205654 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:37.193577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3f4c8-0750-4efa-b08a-65cb7d205654 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:37.194993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3f4c8-0750-4efa-b08a-65cb7d205654 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:37.214989 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 231/464] 10.222.0.13 () {56 vars in 1137 bytes} [Fri Oct 21 21:08:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:37.216005 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3f4c8-0750-4efa-b08a-65cb7d205654 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:37.258017 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087cfb2-b8ec-491b-a9a7-81d4a92c2d39 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:37.293892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362808bd-f210-4278-b2f6-6017eb55bd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:37.294498 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362808bd-f210-4278-b2f6-6017eb55bd50 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:37.294782 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362808bd-f210-4278-b2f6-6017eb55bd50 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:37.295055 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362808bd-f210-4278-b2f6-6017eb55bd50 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:37.295723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362808bd-f210-4278-b2f6-6017eb55bd50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:37.296030 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362808bd-f210-4278-b2f6-6017eb55bd50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:37.297828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362808bd-f210-4278-b2f6-6017eb55bd50 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:37.315096 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 234/465] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:37.316380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362808bd-f210-4278-b2f6-6017eb55bd50 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:37.324078 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3f4c8-0750-4efa-b08a-65cb7d205654 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:37.335786 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:37.362338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924fa164-3b9b-4819-bda8-201a123cbe18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:37.363061 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fa164-3b9b-4819-bda8-201a123cbe18 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:37.363432 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fa164-3b9b-4819-bda8-201a123cbe18 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:37.363827 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fa164-3b9b-4819-bda8-201a123cbe18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:37.364550 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924fa164-3b9b-4819-bda8-201a123cbe18 None admin] RBAC: Adding query filter params (role.id=b0f9a090148d4a0eb6456472edec3d5a, scope.project.id=8b2c33250ef84d79a4a4742fb4252156, user.id=12dad9f016a740b98538722e1b723823) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:37.364855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924fa164-3b9b-4819-bda8-201a123cbe18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:37.366287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924fa164-3b9b-4819-bda8-201a123cbe18 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:37.379608 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 232/466] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:08:37 2022] GET /identity/v3/role_assignments?user.id=12dad9f016a740b98538722e1b723823&scope.project.id=8b2c33250ef84d79a4a4742fb4252156&role.id=b0f9a090148d4a0eb6456472edec3d5a => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:37.379608 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924fa164-3b9b-4819-bda8-201a123cbe18 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:37.386780 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:37.429792 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362808bd-f210-4278-b2f6-6017eb55bd50 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:37.469161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d978f-dc5f-492d-b699-d5d56e8fdf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:37.469810 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d978f-dc5f-492d-b699-d5d56e8fdf10 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:37.470092 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d978f-dc5f-492d-b699-d5d56e8fdf10 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:37.470364 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d978f-dc5f-492d-b699-d5d56e8fdf10 None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:37.471226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d978f-dc5f-492d-b699-d5d56e8fdf10 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:37.471532 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d978f-dc5f-492d-b699-d5d56e8fdf10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:37.473738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d978f-dc5f-492d-b699-d5d56e8fdf10 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:37.494732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 235/467] 10.222.0.13 () {56 vars in 1137 bytes} [Fri Oct 21 21:08:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:37.496544 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d978f-dc5f-492d-b699-d5d56e8fdf10 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:37.525088 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924fa164-3b9b-4819-bda8-201a123cbe18 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:37.584619 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df2512-bea9-4671-9bb5-b7e52f175fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:37.585268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df2512-bea9-4671-9bb5-b7e52f175fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:37.585567 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df2512-bea9-4671-9bb5-b7e52f175fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:37.585836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df2512-bea9-4671-9bb5-b7e52f175fe4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:37.617966 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d978f-dc5f-492d-b699-d5d56e8fdf10 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:37.637344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:37.646727 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:37.668867 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde07df-a78b-4038-80af-d4b02227de95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:37.669619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde07df-a78b-4038-80af-d4b02227de95 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:37.669966 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde07df-a78b-4038-80af-d4b02227de95 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:37.670303 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde07df-a78b-4038-80af-d4b02227de95 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:37.674652 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df2512-bea9-4671-9bb5-b7e52f175fe4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:37.676791 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df2512-bea9-4671-9bb5-b7e52f175fe4 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:37.680743 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde07df-a78b-4038-80af-d4b02227de95 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:37.682305 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde07df-a78b-4038-80af-d4b02227de95 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:37.697729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-9dde07df-a78b-4038-80af-d4b02227de95 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 21 21:08:37.697729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:37.697729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:37.697729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:37.697729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:37.697729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:37.697729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:37.697729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:37.697729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:37.697729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:37.697729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:37.697729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:37.697729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:08:37.697729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:08:37.697729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:08:37.697729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:37.698707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:37.699829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:37.699829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:37.699829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:08:37.699829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:08:37.699829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:08:37.699829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:08:37.699829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:08:37.699829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:08:37.699829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 21 21:08:37.699829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:37.700453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 236/468] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Oct 21 21:08:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:37.704747 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde07df-a78b-4038-80af-d4b02227de95 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:37.805757 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 233/469] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:08:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:08:37.806521 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df2512-bea9-4671-9bb5-b7e52f175fe4 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:37.820972 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde07df-a78b-4038-80af-d4b02227de95 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:37.856387 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4e62f-4e9c-4ce3-8109-d37afd811505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:37.857177 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4e62f-4e9c-4ce3-8109-d37afd811505 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:37.857752 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4e62f-4e9c-4ce3-8109-d37afd811505 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:37.858124 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4e62f-4e9c-4ce3-8109-d37afd811505 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:37.902861 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:37.924862 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:37.930133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df2512-bea9-4671-9bb5-b7e52f175fe4 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:37.951525 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4e62f-4e9c-4ce3-8109-d37afd811505 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:37.953452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4e62f-4e9c-4ce3-8109-d37afd811505 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:37.964895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfba3b6-f43d-4907-87ef-051b07fe318d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:37.965509 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfba3b6-f43d-4907-87ef-051b07fe318d None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:37.966074 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfba3b6-f43d-4907-87ef-051b07fe318d None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:37.966074 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfba3b6-f43d-4907-87ef-051b07fe318d None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:37.966752 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfba3b6-f43d-4907-87ef-051b07fe318d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:37.967051 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfba3b6-f43d-4907-87ef-051b07fe318d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:37.968538 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfba3b6-f43d-4907-87ef-051b07fe318d None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:37.986118 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 234/470] 10.222.0.13 () {56 vars in 1139 bytes} [Fri Oct 21 21:08:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:37.986940 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfba3b6-f43d-4907-87ef-051b07fe318d None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:38.079060 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 237/471] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:08:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:08:38.084103 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4e62f-4e9c-4ce3-8109-d37afd811505 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:38.097676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfba3b6-f43d-4907-87ef-051b07fe318d None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:38.126914 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:38.132454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc14c2-c2cd-4c3b-851b-558b7d0aaa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:38.133313 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc14c2-c2cd-4c3b-851b-558b7d0aaa03 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:38.133822 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc14c2-c2cd-4c3b-851b-558b7d0aaa03 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:38.134250 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc14c2-c2cd-4c3b-851b-558b7d0aaa03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:38.145555 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc14c2-c2cd-4c3b-851b-558b7d0aaa03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:38.147106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc14c2-c2cd-4c3b-851b-558b7d0aaa03 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:38.162231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-f3dc14c2-c2cd-4c3b-851b-558b7d0aaa03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:38.162231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:38.162231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:38.162231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:38.162231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:38.162231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:38.162231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:38.162231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:38.162231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:38.162231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:38.162231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:38.162231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:38.162231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:38.162231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:38.162231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:38.162231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:38.163596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:38.163596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:38.163596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:38.163596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:38.163596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:38.163596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:38.163596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:38.163596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:38.163596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:38.163596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:38.163596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:38.163596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:38.163596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:38.163596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:38.163596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:38.163596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:38.163596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:38.164443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:38.164443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:38.164443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:38.164443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 235/472] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:38.164443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc14c2-c2cd-4c3b-851b-558b7d0aaa03 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:38.172634 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:38.194099 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4e62f-4e9c-4ce3-8109-d37afd811505 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:38.232871 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927e61b-c941-4c65-955c-a849f5abdac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:38.233796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927e61b-c941-4c65-955c-a849f5abdac4 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:38.234276 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927e61b-c941-4c65-955c-a849f5abdac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:38.234683 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927e61b-c941-4c65-955c-a849f5abdac4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:38.274676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc14c2-c2cd-4c3b-851b-558b7d0aaa03 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:38.309096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9cd06-431a-4de4-a1ed-e088efcaed51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:38.309729 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9cd06-431a-4de4-a1ed-e088efcaed51 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:38.310007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9cd06-431a-4de4-a1ed-e088efcaed51 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:38.310277 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9cd06-431a-4de4-a1ed-e088efcaed51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:38.310866 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927e61b-c941-4c65-955c-a849f5abdac4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:38.312702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927e61b-c941-4c65-955c-a849f5abdac4 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:38.321626 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9cd06-431a-4de4-a1ed-e088efcaed51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:38.323232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9cd06-431a-4de4-a1ed-e088efcaed51 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:38.338338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-aee9cd06-431a-4de4-a1ed-e088efcaed51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:38.338338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:38.338338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:38.338338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:38.338338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:38.338338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:38.338338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:38.338338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:38.338338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:38.338338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:38.338338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:38.338338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:38.338338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:38.338338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:38.338338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:38.338338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:38.339226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:38.339226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:38.339226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:38.339226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:38.339226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:38.339226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:38.339226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:38.339226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:38.339226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:38.339226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:38.339226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:38.339226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:38.339226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:38.339226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:38.339226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:38.339226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:38.339226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:38.340082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:38.340082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:38.340082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:38.344237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 236/473] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:38.345186 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9cd06-431a-4de4-a1ed-e088efcaed51 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:38.381253 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:38.405344 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:38.430171 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 238/474] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:08:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:08:38.431276 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927e61b-c941-4c65-955c-a849f5abdac4 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:38.451479 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9cd06-431a-4de4-a1ed-e088efcaed51 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:38.486893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c5f44-1e24-4276-9094-141bb7454129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:38.487737 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c5f44-1e24-4276-9094-141bb7454129 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:38.488210 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c5f44-1e24-4276-9094-141bb7454129 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:38.488558 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c5f44-1e24-4276-9094-141bb7454129 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:38.489272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c5f44-1e24-4276-9094-141bb7454129 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:38.489658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c5f44-1e24-4276-9094-141bb7454129 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:38.491079 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c5f44-1e24-4276-9094-141bb7454129 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:38.507683 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 237/475] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:38.508869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c5f44-1e24-4276-9094-141bb7454129 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:38.541252 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927e61b-c941-4c65-955c-a849f5abdac4 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:38.576728 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6b25e-962f-4724-8f65-84de720bf37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:38.577676 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6b25e-962f-4724-8f65-84de720bf37f None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:38.578309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6b25e-962f-4724-8f65-84de720bf37f None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:38.578769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6b25e-962f-4724-8f65-84de720bf37f None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:38.579690 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6b25e-962f-4724-8f65-84de720bf37f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:38.580179 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6b25e-962f-4724-8f65-84de720bf37f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:38.582295 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6b25e-962f-4724-8f65-84de720bf37f None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:38.595071 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 239/476] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:38.596164 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6b25e-962f-4724-8f65-84de720bf37f None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:38.609134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:38.614994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c5f44-1e24-4276-9094-141bb7454129 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:38.645142 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:38.651038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9c9f3-1b9a-44dd-b7b4-eacb054fff83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:38.651824 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9c9f3-1b9a-44dd-b7b4-eacb054fff83 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:38.652200 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9c9f3-1b9a-44dd-b7b4-eacb054fff83 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:38.652573 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9c9f3-1b9a-44dd-b7b4-eacb054fff83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:38.663986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9c9f3-1b9a-44dd-b7b4-eacb054fff83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:38.665606 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9c9f3-1b9a-44dd-b7b4-eacb054fff83 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:38.681367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-d7d9c9f3-1b9a-44dd-b7b4-eacb054fff83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:38.681367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:38.681367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:38.681367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:38.681367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:38.681367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:38.681367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:38.681367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:38.681367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:38.681367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:38.681367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:38.681367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:38.681367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:38.681367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:38.681367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:38.681367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:38.682413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:38.682413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:38.682413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:38.682413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:38.682413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:38.682413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:38.682413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:38.682413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:38.682413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:38.682413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:38.682413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:38.682413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:38.682413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:38.682413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:38.682413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:38.682413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:38.682413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:38.683223 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:38.683223 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:38.683223 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:38.683696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 238/477] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:38.684357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9c9f3-1b9a-44dd-b7b4-eacb054fff83 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:38.713579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6b25e-962f-4724-8f65-84de720bf37f None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:38.754150 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff00e7d-d6b5-4a44-a091-83d149e9243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:38.755239 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff00e7d-d6b5-4a44-a091-83d149e9243e None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:38.755738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff00e7d-d6b5-4a44-a091-83d149e9243e None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:38.756320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff00e7d-d6b5-4a44-a091-83d149e9243e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:38.764847 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff00e7d-d6b5-4a44-a091-83d149e9243e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:38.766666 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff00e7d-d6b5-4a44-a091-83d149e9243e None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:38.779719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-0ff00e7d-d6b5-4a44-a091-83d149e9243e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:38.779719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:38.779719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:38.779719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:38.779719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:38.779719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:38.779719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:38.779719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:38.779719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:38.779719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:38.779719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:38.779719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:38.779719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:08:38.779719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:08:38.779719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:08:38.780630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:08:38.780630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:38.780630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:38.780630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:38.780630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:38.780630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:38.780630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:38.780630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:38.780630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:38.780630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:38.780630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:38.780630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:38.780630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:38.780630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:38.780630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:38.780630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:38.780630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:38.781463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:08:38.781463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:08:38.781463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:08:38.781463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:08:38.781463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:08:38.781463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:08:38.781463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:38.781463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:38.783930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 240/478] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:38.784934 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff00e7d-d6b5-4a44-a091-83d149e9243e None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:38.804715 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9c9f3-1b9a-44dd-b7b4-eacb054fff83 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:38.839655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a30c89-d51e-433e-9a73-b8bceb6ad7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:38.840291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a30c89-d51e-433e-9a73-b8bceb6ad7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:38.840572 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a30c89-d51e-433e-9a73-b8bceb6ad7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:38.840844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a30c89-d51e-433e-9a73-b8bceb6ad7b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:38.848326 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a30c89-d51e-433e-9a73-b8bceb6ad7b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:38.849853 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a30c89-d51e-433e-9a73-b8bceb6ad7b5 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:38.861528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-b8a30c89-d51e-433e-9a73-b8bceb6ad7b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:38.861528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:38.861528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:38.861528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:38.861528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:38.861528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:38.861528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:38.861528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:38.861528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:38.861528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:38.861528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:38.861528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:38.861528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:08:38.861528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:08:38.861528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:08:38.862582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:08:38.862582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:38.862582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:38.862582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:38.862582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:38.862582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:38.862582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:38.862582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:38.862582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:38.862582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:38.862582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:38.862582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:38.862582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:38.862582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:38.862582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:38.862582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:38.862582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:38.865360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:08:38.865360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:08:38.865360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:08:38.865360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:08:38.865360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:08:38.865360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:08:38.865360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:38.865360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:38.865360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 239/479] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:38.865360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a30c89-d51e-433e-9a73-b8bceb6ad7b5 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:38.878587 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:38.899361 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff00e7d-d6b5-4a44-a091-83d149e9243e None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:38.901117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:38.935560 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3ebbe3-337a-43a0-8c3f-c9dc65cbfca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:38.936174 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ebbe3-337a-43a0-8c3f-c9dc65cbfca1 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:38.936401 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ebbe3-337a-43a0-8c3f-c9dc65cbfca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:38.936669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ebbe3-337a-43a0-8c3f-c9dc65cbfca1 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:38.937291 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3ebbe3-337a-43a0-8c3f-c9dc65cbfca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:38.937589 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3ebbe3-337a-43a0-8c3f-c9dc65cbfca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:38.939020 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3ebbe3-337a-43a0-8c3f-c9dc65cbfca1 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:38.956161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 241/480] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:38.956161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fec46c-3df8-4db9-9229-2bf8ddfedc15 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:38.956161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fec46c-3df8-4db9-9229-2bf8ddfedc15 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:38.956161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fec46c-3df8-4db9-9229-2bf8ddfedc15 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:38.956161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 242/481] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:38.956161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fec46c-3df8-4db9-9229-2bf8ddfedc15 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:38.979503 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a30c89-d51e-433e-9a73-b8bceb6ad7b5 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:39.014233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b920f-cf40-4977-84fc-f174de59b961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:39.014840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b920f-cf40-4977-84fc-f174de59b961 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:39.015146 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b920f-cf40-4977-84fc-f174de59b961 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:39.015394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b920f-cf40-4977-84fc-f174de59b961 None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:39.016109 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b920f-cf40-4977-84fc-f174de59b961 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:39.016414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b920f-cf40-4977-84fc-f174de59b961 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:39.017831 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b920f-cf40-4977-84fc-f174de59b961 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:39.034659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 240/482] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:08:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:39.034659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b920f-cf40-4977-84fc-f174de59b961 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:39.063104 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fec46c-3df8-4db9-9229-2bf8ddfedc15 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:39.100117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cccde64-56ea-46cb-8ecb-c950869666b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:39.100762 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccde64-56ea-46cb-8ecb-c950869666b1 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:39.101040 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccde64-56ea-46cb-8ecb-c950869666b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:39.101308 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccde64-56ea-46cb-8ecb-c950869666b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:39.101987 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cccde64-56ea-46cb-8ecb-c950869666b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:39.102287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cccde64-56ea-46cb-8ecb-c950869666b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:39.103807 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cccde64-56ea-46cb-8ecb-c950869666b1 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:39.108965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:39.120908 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 243/483] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:08:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:39.125653 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4320c5e-0f04-4f27-a020-54a6c7cd6a7d None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:39.125940 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4320c5e-0f04-4f27-a020-54a6c7cd6a7d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:39.126358 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4320c5e-0f04-4f27-a020-54a6c7cd6a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:39.143361 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b920f-cf40-4977-84fc-f174de59b961 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:39.144720 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-d4320c5e-0f04-4f27-a020-54a6c7cd6a7d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:39.145891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d4320c5e-0f04-4f27-a020-54a6c7cd6a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:39.162838 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:39.181214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70b878-2ce4-43bd-a345-dfc5dd2c20f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:39.181844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70b878-2ce4-43bd-a345-dfc5dd2c20f4 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:39.182123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70b878-2ce4-43bd-a345-dfc5dd2c20f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:39.182396 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70b878-2ce4-43bd-a345-dfc5dd2c20f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:39.189339 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70b878-2ce4-43bd-a345-dfc5dd2c20f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:39.190828 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70b878-2ce4-43bd-a345-dfc5dd2c20f4 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:39.202103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-bf70b878-2ce4-43bd-a345-dfc5dd2c20f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:39.202103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:39.202103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:39.202103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:39.202103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:39.202103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:39.202103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:39.202103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:39.202103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:39.202103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:39.202103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:39.202103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:39.202103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:08:39.202103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:08:39.202103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:08:39.203107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:08:39.203107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:39.203107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:39.203107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:39.203107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:39.203107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:39.203107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:39.203107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:39.203107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:39.203107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:39.203107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:39.203107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:39.203107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:39.203107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:39.203107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:39.203107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:39.203107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:39.204601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:08:39.204601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:08:39.204601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:08:39.204601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:08:39.204601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:08:39.204601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:08:39.204601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:39.204601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:39.204601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 241/484] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:39.209033 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70b878-2ce4-43bd-a345-dfc5dd2c20f4 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:39.227900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 244/485] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:39.227900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4320c5e-0f04-4f27-a020-54a6c7cd6a7d None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:39.319304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70b878-2ce4-43bd-a345-dfc5dd2c20f4 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:39.346022 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4320c5e-0f04-4f27-a020-54a6c7cd6a7d None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:39.346794 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:39.356393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34bf70-115a-4a77-be10-372eddf8a1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:39.357346 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34bf70-115a-4a77-be10-372eddf8a1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:39.357625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34bf70-115a-4a77-be10-372eddf8a1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:39.357896 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34bf70-115a-4a77-be10-372eddf8a1f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:39.359019 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34bf70-115a-4a77-be10-372eddf8a1f8 None admin] RBAC: Adding query filter params (role.id=b0f9a090148d4a0eb6456472edec3d5a, scope.project.id=8b2c33250ef84d79a4a4742fb4252156, user.id=d620cad0ef21453a9cab61f728913cdd) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:39.359545 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34bf70-115a-4a77-be10-372eddf8a1f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:39.361344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34bf70-115a-4a77-be10-372eddf8a1f8 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:39.369165 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 242/486] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:08:39 2022] GET /identity/v3/role_assignments?user.id=d620cad0ef21453a9cab61f728913cdd&scope.project.id=8b2c33250ef84d79a4a4742fb4252156&role.id=b0f9a090148d4a0eb6456472edec3d5a => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:39.370038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34bf70-115a-4a77-be10-372eddf8a1f8 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:39.419634 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531dd354-bcfc-481b-af0f-9bcd2c65d0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:39.420488 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dd354-bcfc-481b-af0f-9bcd2c65d0ae None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:39.420888 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dd354-bcfc-481b-af0f-9bcd2c65d0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:39.421261 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dd354-bcfc-481b-af0f-9bcd2c65d0ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:39.422094 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531dd354-bcfc-481b-af0f-9bcd2c65d0ae None admin] RBAC: Adding query filter params (role.id=b0f9a090148d4a0eb6456472edec3d5a, scope.project.id=8b2c33250ef84d79a4a4742fb4252156, user.id=5ce5ec441b06408bbadd6fc040af0da5) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:39.422503 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531dd354-bcfc-481b-af0f-9bcd2c65d0ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:39.424149 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531dd354-bcfc-481b-af0f-9bcd2c65d0ae None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:39.442140 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 245/487] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:08:39 2022] GET /identity/v3/role_assignments?user.id=5ce5ec441b06408bbadd6fc040af0da5&scope.project.id=8b2c33250ef84d79a4a4742fb4252156&role.id=b0f9a090148d4a0eb6456472edec3d5a => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:39.512197 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531dd354-bcfc-481b-af0f-9bcd2c65d0ae None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:39.522649 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34bf70-115a-4a77-be10-372eddf8a1f8 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:39.561802 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:39.568481 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd670a-4cf0-4406-a620-a84632b32a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:39.569312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd670a-4cf0-4406-a620-a84632b32a6c None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:39.569592 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd670a-4cf0-4406-a620-a84632b32a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:39.569859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd670a-4cf0-4406-a620-a84632b32a6c None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:39.571376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd670a-4cf0-4406-a620-a84632b32a6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:39.571789 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd670a-4cf0-4406-a620-a84632b32a6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:39.573491 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd670a-4cf0-4406-a620-a84632b32a6c None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:39.599643 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 243/488] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:08:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:39.606006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd670a-4cf0-4406-a620-a84632b32a6c None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:39.669282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:39.724551 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531dd354-bcfc-481b-af0f-9bcd2c65d0ae None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:39.748990 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd670a-4cf0-4406-a620-a84632b32a6c None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:39.767958 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f535430-2030-45fd-9f9e-a51a84c317d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:39.769287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f535430-2030-45fd-9f9e-a51a84c317d7 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:39.769571 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f535430-2030-45fd-9f9e-a51a84c317d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:39.769836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f535430-2030-45fd-9f9e-a51a84c317d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:39.782231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f535430-2030-45fd-9f9e-a51a84c317d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:39.787487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-0f535430-2030-45fd-9f9e-a51a84c317d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:39.787487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:39.787487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:39.787487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:39.787487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:39.787487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:39.787487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:39.787487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:39.787487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:39.787487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:39.787487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:39.787487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:39.787487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:39.787487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:39.787487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:39.787487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:39.788573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:39.788573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:39.788573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:39.788573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:39.788573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:39.788573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:39.788573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:39.788573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:39.788573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:39.788573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:39.788573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:39.788573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:39.788573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:39.788573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:39.788573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:39.788573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:39.789547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:39.789547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:39.789547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:39.789547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:39.789547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:39.789547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:39.789547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:39.789547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:39.789547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:39.789547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:39.789547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:39.789547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:39.789547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 246/489] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Oct 21 21:08:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:39.790269 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb899b2-6150-4cf5-bfdf-3dadcecf6f78 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:39.790347 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb899b2-6150-4cf5-bfdf-3dadcecf6f78 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:39.790601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb899b2-6150-4cf5-bfdf-3dadcecf6f78 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:39.791821 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8640d7a-cd89-4299-9fe2-8f0cf3b731e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:39.792276 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 247/490] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:39.792758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8640d7a-cd89-4299-9fe2-8f0cf3b731e6 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:39.793043 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8640d7a-cd89-4299-9fe2-8f0cf3b731e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:39.793317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8640d7a-cd89-4299-9fe2-8f0cf3b731e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:39.794047 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8640d7a-cd89-4299-9fe2-8f0cf3b731e6 None admin] RBAC: Adding query filter params (role.id=b0f9a090148d4a0eb6456472edec3d5a, scope.project.id=8b2c33250ef84d79a4a4742fb4252156, user.id=d34dc4e7c5474b6b96977019328d8890) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:39.794358 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8640d7a-cd89-4299-9fe2-8f0cf3b731e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:39.796349 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb899b2-6150-4cf5-bfdf-3dadcecf6f78 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:39.800449 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8640d7a-cd89-4299-9fe2-8f0cf3b731e6 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:39.807480 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 244/491] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:08:39 2022] GET /identity/v3/role_assignments?user.id=d34dc4e7c5474b6b96977019328d8890&scope.project.id=8b2c33250ef84d79a4a4742fb4252156&role.id=b0f9a090148d4a0eb6456472edec3d5a => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:39.808247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df812077-e13a-4d51-bda9-c0bc7ea5b2ee None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:39.808540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df812077-e13a-4d51-bda9-c0bc7ea5b2ee None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:39.808816 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df812077-e13a-4d51-bda9-c0bc7ea5b2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:39.830008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-df812077-e13a-4d51-bda9-c0bc7ea5b2ee None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:39.831799 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-df812077-e13a-4d51-bda9-c0bc7ea5b2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:39.907846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:39.923066 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 245/492] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:39.949509 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb899b2-6150-4cf5-bfdf-3dadcecf6f78 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:39.998872 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4dcbd-b0bd-4df7-9479-88a495c4b22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:39.999809 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4dcbd-b0bd-4df7-9479-88a495c4b22c None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:40.000260 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4dcbd-b0bd-4df7-9479-88a495c4b22c None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:40.000701 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4dcbd-b0bd-4df7-9479-88a495c4b22c None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:40.001519 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4dcbd-b0bd-4df7-9479-88a495c4b22c None admin] RBAC: Adding query filter params (name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:40.001986 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4dcbd-b0bd-4df7-9479-88a495c4b22c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:40.003653 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4dcbd-b0bd-4df7-9479-88a495c4b22c None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:40.013232 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 248/493] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Oct 21 21:08:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:40.019605 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df812077-e13a-4d51-bda9-c0bc7ea5b2ee None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:40.071318 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:40.126498 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df812077-e13a-4d51-bda9-c0bc7ea5b2ee None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:40.163118 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91541ef8-1797-46aa-aa45-447621230841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:40.163781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91541ef8-1797-46aa-aa45-447621230841 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:40.164057 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91541ef8-1797-46aa-aa45-447621230841 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:40.164378 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91541ef8-1797-46aa-aa45-447621230841 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:40.174877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91541ef8-1797-46aa-aa45-447621230841 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:40.176331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91541ef8-1797-46aa-aa45-447621230841 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:40.191603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-91541ef8-1797-46aa-aa45-447621230841 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:40.191603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:40.191603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:40.191603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:40.191603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:40.191603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:40.191603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:40.191603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:40.191603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:40.191603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:40.191603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:40.191603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:40.191603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:40.191603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:40.191603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:40.191603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:40.192907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:40.192907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:40.192907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:40.192907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:40.192907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:40.192907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:40.192907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:40.192907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:40.192907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:40.192907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:40.192907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:40.192907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:40.192907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:40.192907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:40.192907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:40.192907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:40.192907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:40.194264 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:40.194264 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:40.194264 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:40.194264 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 246/494] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:08:40.200757 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4dcbd-b0bd-4df7-9479-88a495c4b22c None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:40.314578 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4dcbd-b0bd-4df7-9479-88a495c4b22c None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:40.349117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85922b-a909-4b18-941b-642ec250293d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:40.349724 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85922b-a909-4b18-941b-642ec250293d None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:40.349995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85922b-a909-4b18-941b-642ec250293d None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:40.350262 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85922b-a909-4b18-941b-642ec250293d None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:40.350897 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85922b-a909-4b18-941b-642ec250293d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:40.351198 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85922b-a909-4b18-941b-642ec250293d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:40.352679 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85922b-a909-4b18-941b-642ec250293d None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:40.359170 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:40.365103 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 249/495] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:40.372621 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91541ef8-1797-46aa-aa45-447621230841 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:40.379993 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85922b-a909-4b18-941b-642ec250293d None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:40.531706 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:40.536173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91541ef8-1797-46aa-aa45-447621230841 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:40.571855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dac26-d1a4-489e-993f-635c86e8cefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:40.573560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dac26-d1a4-489e-993f-635c86e8cefa None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:40.573560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dac26-d1a4-489e-993f-635c86e8cefa None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:40.573560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dac26-d1a4-489e-993f-635c86e8cefa None admin] PATH_INFO: `/v3/users/swift` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:40.576990 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85922b-a909-4b18-941b-642ec250293d None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:40.587595 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15dac26-d1a4-489e-993f-635c86e8cefa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:40.587595 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15dac26-d1a4-489e-993f-635c86e8cefa None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:40.605924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-c15dac26-d1a4-489e-993f-635c86e8cefa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 21 21:08:40.605924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:40.605924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:40.605924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:40.605924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:40.605924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:40.605924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:40.605924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:40.605924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:40.605924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:40.605924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:40.605924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:40.605924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:08:40.605924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:08:40.605924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:08:40.605924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:40.606886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:40.607818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:40.607818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:40.607818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:08:40.607818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:08:40.607818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:08:40.607818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:08:40.607818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:08:40.607818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:08:40.607818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 21 21:08:40.607818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:40.607818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 247/496] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:08:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:40.613422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dac26-d1a4-489e-993f-635c86e8cefa None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:40.654869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a6faf-57da-437d-b599-c55ec7a8bfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:40.655511 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a6faf-57da-437d-b599-c55ec7a8bfbb None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:40.655814 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a6faf-57da-437d-b599-c55ec7a8bfbb None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:40.656086 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a6faf-57da-437d-b599-c55ec7a8bfbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:40.677951 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a6faf-57da-437d-b599-c55ec7a8bfbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:40.683445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-7f2a6faf-57da-437d-b599-c55ec7a8bfbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:08:40.683445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:40.683445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:40.683445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:40.683445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:40.683445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:40.683445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:40.683445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:40.683445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:40.683445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:40.683445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:40.683445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:40.683445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:40.683445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:40.683445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:40.683445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:40.684363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:40.684363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:40.684363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:40.684363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:40.684363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:40.684363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:40.684363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:40.684363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:40.684363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:40.684363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:40.684363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:40.684363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:40.684363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:40.684363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:40.684363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:40.684363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:40.685180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:40.685180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:40.685180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:40.685180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:40.685180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:40.685180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:40.685180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:40.685180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:40.685180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:40.685180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:40.685180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:08:40.685180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:40.685180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 250/497] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:08:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:08:40.691832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a6faf-57da-437d-b599-c55ec7a8bfbb None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:40.716904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dac26-d1a4-489e-993f-635c86e8cefa None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:40.754131 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1d20f-f5e2-4a83-bd8e-d1e184cff359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:40.754751 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1d20f-f5e2-4a83-bd8e-d1e184cff359 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:40.755033 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1d20f-f5e2-4a83-bd8e-d1e184cff359 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:40.755297 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1d20f-f5e2-4a83-bd8e-d1e184cff359 None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:40.756061 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1d20f-f5e2-4a83-bd8e-d1e184cff359 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:40.756370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1d20f-f5e2-4a83-bd8e-d1e184cff359 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:40.757794 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1d20f-f5e2-4a83-bd8e-d1e184cff359 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:40.780885 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 248/498] 10.222.0.13 () {56 vars in 1135 bytes} [Fri Oct 21 21:08:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:40.782255 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1d20f-f5e2-4a83-bd8e-d1e184cff359 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:40.784149 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:40.802230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:40.870362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a6faf-57da-437d-b599-c55ec7a8bfbb None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:40.887434 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1d20f-f5e2-4a83-bd8e-d1e184cff359 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:40.924191 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0bef2-8ae5-4719-825c-1355e57ff7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:40.924882 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0bef2-8ae5-4719-825c-1355e57ff7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:40.925676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0bef2-8ae5-4719-825c-1355e57ff7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:40.925961 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0bef2-8ae5-4719-825c-1355e57ff7a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:40.953202 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815ee68-7f96-4547-8462-0df88149941f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:40.953812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815ee68-7f96-4547-8462-0df88149941f None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:40.955116 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815ee68-7f96-4547-8462-0df88149941f None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:40.955393 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815ee68-7f96-4547-8462-0df88149941f None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:40.956645 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815ee68-7f96-4547-8462-0df88149941f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:40.956954 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815ee68-7f96-4547-8462-0df88149941f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:40.958347 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815ee68-7f96-4547-8462-0df88149941f None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:40.973666 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 251/499] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:40.979080 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815ee68-7f96-4547-8462-0df88149941f None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:41.040747 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0bef2-8ae5-4719-825c-1355e57ff7a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:41.042264 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0bef2-8ae5-4719-825c-1355e57ff7a4 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:41.077054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:41.144818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815ee68-7f96-4547-8462-0df88149941f None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:41.165766 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:41.181221 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8907b-e2da-4dc4-b290-e66c1a03aa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:41.182082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8907b-e2da-4dc4-b290-e66c1a03aa41 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:41.182543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8907b-e2da-4dc4-b290-e66c1a03aa41 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:41.182998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8907b-e2da-4dc4-b290-e66c1a03aa41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:41.186335 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 249/500] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:08:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 21:08:41.192039 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0bef2-8ae5-4719-825c-1355e57ff7a4 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:41.195742 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8907b-e2da-4dc4-b290-e66c1a03aa41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:41.197416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8907b-e2da-4dc4-b290-e66c1a03aa41 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:41.214123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-42b8907b-e2da-4dc4-b290-e66c1a03aa41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:41.214123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:41.214123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:41.214123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:41.214123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:41.214123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:41.214123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:41.214123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:41.214123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:41.214123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:41.214123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:41.214123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:41.214123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:41.214123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:41.214123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:41.214123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:41.215723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:41.215723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:41.215723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:41.215723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:41.215723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:41.215723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:41.215723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:41.215723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:41.215723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:41.215723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:41.215723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:41.215723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:41.215723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:41.215723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:41.215723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:41.215723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:41.215723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:41.216813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:41.216813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:41.216813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:41.223979 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 252/501] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:08:41.229347 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8907b-e2da-4dc4-b290-e66c1a03aa41 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:41.378857 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8907b-e2da-4dc4-b290-e66c1a03aa41 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:41.397568 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0bef2-8ae5-4719-825c-1355e57ff7a4 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:41.418441 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d7b71-126e-4c53-936f-367afe3e3c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:41.419431 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d7b71-126e-4c53-936f-367afe3e3c9e None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:41.419889 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d7b71-126e-4c53-936f-367afe3e3c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:41.420371 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d7b71-126e-4c53-936f-367afe3e3c9e None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:41.422439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d7b71-126e-4c53-936f-367afe3e3c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:41.422880 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d7b71-126e-4c53-936f-367afe3e3c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:41.424573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d7b71-126e-4c53-936f-367afe3e3c9e None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:41.435456 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:41.440980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 253/502] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:41.447116 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d7b71-126e-4c53-936f-367afe3e3c9e None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:41.488443 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:41.520849 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff4592-6116-4a24-9077-7e7e76dc8316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:41.521771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff4592-6116-4a24-9077-7e7e76dc8316 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:41.521771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff4592-6116-4a24-9077-7e7e76dc8316 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:41.522027 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff4592-6116-4a24-9077-7e7e76dc8316 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:41.545496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff4592-6116-4a24-9077-7e7e76dc8316 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:41.546944 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff4592-6116-4a24-9077-7e7e76dc8316 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:41.557618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d7b71-126e-4c53-936f-367afe3e3c9e None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:41.579133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-b6ff4592-6116-4a24-9077-7e7e76dc8316 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:41.579133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:41.579133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:41.579133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:41.579133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:41.579133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:41.579133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:41.579133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:41.579133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:41.579133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:41.579133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:41.579133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:41.579133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:41.579133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:41.579133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:41.579133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:41.580266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:41.580266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:41.580266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:41.580266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:41.580266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:41.580266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:41.580266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:41.580266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:41.580266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:41.580266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:41.580266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:41.580266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:41.580266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:41.580266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:41.580266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:41.580266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:41.580266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:41.581712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:41.581712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:41.581712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:41.582096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 250/503] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:08:41.592699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff4592-6116-4a24-9077-7e7e76dc8316 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:41.595977 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d651c9f-1599-4912-957e-809614d293be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:41.596994 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d651c9f-1599-4912-957e-809614d293be None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:41.597422 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d651c9f-1599-4912-957e-809614d293be None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:41.597941 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d651c9f-1599-4912-957e-809614d293be None admin] PATH_INFO: `/v3/users/nova` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:41.606354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d651c9f-1599-4912-957e-809614d293be None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:41.608144 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d651c9f-1599-4912-957e-809614d293be None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:41.620812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-7d651c9f-1599-4912-957e-809614d293be None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 21 21:08:41.620812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:41.620812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:41.620812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:41.620812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:41.620812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:41.620812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:41.620812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:41.620812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:41.620812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:41.620812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:41.620812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:41.620812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:08:41.620812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:08:41.620812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:08:41.620812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:41.621964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:41.623159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:41.623159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:41.623159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:08:41.623159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:08:41.623159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:08:41.623159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:08:41.623159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:08:41.623159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:08:41.623159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 21 21:08:41.623159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:41.625439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 254/504] 10.222.0.13 () {56 vars in 1093 bytes} [Fri Oct 21 21:08:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:41.631496 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d651c9f-1599-4912-957e-809614d293be None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:41.744965 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d651c9f-1599-4912-957e-809614d293be None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:41.747798 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:41.808576 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc73e85-58e9-4b41-a4f8-0f96daa39f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:41.809236 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc73e85-58e9-4b41-a4f8-0f96daa39f75 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:41.809460 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc73e85-58e9-4b41-a4f8-0f96daa39f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:41.809725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc73e85-58e9-4b41-a4f8-0f96daa39f75 None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:41.810455 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc73e85-58e9-4b41-a4f8-0f96daa39f75 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:41.810765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc73e85-58e9-4b41-a4f8-0f96daa39f75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:41.817797 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc73e85-58e9-4b41-a4f8-0f96daa39f75 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:41.829940 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff4592-6116-4a24-9077-7e7e76dc8316 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:41.856800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 255/505] 10.222.0.13 () {56 vars in 1133 bytes} [Fri Oct 21 21:08:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:41.859370 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc73e85-58e9-4b41-a4f8-0f96daa39f75 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:41.899560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5c89c-15c0-43f6-b1ca-ea9e3154b0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:41.900196 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5c89c-15c0-43f6-b1ca-ea9e3154b0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:41.900479 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5c89c-15c0-43f6-b1ca-ea9e3154b0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:41.900749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5c89c-15c0-43f6-b1ca-ea9e3154b0a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:41.901400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5c89c-15c0-43f6-b1ca-ea9e3154b0a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:41.901707 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5c89c-15c0-43f6-b1ca-ea9e3154b0a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:41.903077 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5c89c-15c0-43f6-b1ca-ea9e3154b0a1 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:41.924125 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 251/506] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:41.930173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5c89c-15c0-43f6-b1ca-ea9e3154b0a1 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:41.974102 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:42.054674 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5c89c-15c0-43f6-b1ca-ea9e3154b0a1 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:42.090590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc73e85-58e9-4b41-a4f8-0f96daa39f75 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:42.093895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d98c95-64ba-44e1-b9b5-4ddb45d5d7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:42.094128 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d98c95-64ba-44e1-b9b5-4ddb45d5d7a1 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:42.094415 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d98c95-64ba-44e1-b9b5-4ddb45d5d7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:42.094691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d98c95-64ba-44e1-b9b5-4ddb45d5d7a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:42.102079 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d98c95-64ba-44e1-b9b5-4ddb45d5d7a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:42.103670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d98c95-64ba-44e1-b9b5-4ddb45d5d7a1 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:42.117437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-b3d98c95-64ba-44e1-b9b5-4ddb45d5d7a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:42.117437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:42.117437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:42.117437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:42.117437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:42.117437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:42.117437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:42.117437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:42.117437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:42.117437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:42.117437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:42.117437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:42.117437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:08:42.117437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:08:42.117437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:08:42.118394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:08:42.118394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:42.118394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:42.118394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:42.118394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:42.118394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:42.118394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:42.118394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:42.118394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:42.118394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:42.118394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:42.118394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:42.118394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:42.118394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:42.118394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:42.118394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:42.118394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:42.119247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:08:42.119247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:08:42.119247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:08:42.119247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:08:42.119247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:08:42.119247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:08:42.119247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:42.119247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:42.119247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 252/507] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:42.120272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f78199-7648-4f74-8bc6-de1e67f468ce None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:42.120272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f78199-7648-4f74-8bc6-de1e67f468ce None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:42.120397 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f78199-7648-4f74-8bc6-de1e67f468ce None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:42.126014 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 253/508] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:42.126014 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f78199-7648-4f74-8bc6-de1e67f468ce None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:42.143200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfc992-4504-4338-94de-737c9ec646aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:42.145390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfc992-4504-4338-94de-737c9ec646aa None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:42.146176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfc992-4504-4338-94de-737c9ec646aa None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:42.146674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfc992-4504-4338-94de-737c9ec646aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:42.161534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfc992-4504-4338-94de-737c9ec646aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:42.164131 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfc992-4504-4338-94de-737c9ec646aa None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:42.182769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-25bfc992-4504-4338-94de-737c9ec646aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:42.182769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:42.182769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:42.182769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:42.182769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:42.182769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:42.182769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:42.182769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:42.182769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:42.182769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:42.182769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:42.182769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:42.182769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:42.182769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:42.182769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:42.182769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:42.183769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:42.183769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:42.183769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:42.183769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:42.183769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:42.183769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:42.183769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:42.183769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:42.183769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:42.183769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:42.183769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:42.183769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:42.183769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:42.183769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:42.183769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:42.183769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:42.183769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:42.184635 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:42.184635 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:42.184635 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:42.184635 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 256/509] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:42.185288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e350f7-b3d5-4894-9655-ff5379e06ff6 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:42.185382 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e350f7-b3d5-4894-9655-ff5379e06ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:42.185617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e350f7-b3d5-4894-9655-ff5379e06ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:42.191880 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:42.205776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-07e350f7-b3d5-4894-9655-ff5379e06ff6 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:42.207490 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-07e350f7-b3d5-4894-9655-ff5379e06ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:42.242655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:42.258092 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f78199-7648-4f74-8bc6-de1e67f468ce None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:42.280165 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 257/510] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:42.281003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e350f7-b3d5-4894-9655-ff5379e06ff6 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:42.301080 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76278523-a092-42c7-9ace-2ae739fedff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:42.302379 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76278523-a092-42c7-9ace-2ae739fedff2 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:42.302854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76278523-a092-42c7-9ace-2ae739fedff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:42.303319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76278523-a092-42c7-9ace-2ae739fedff2 None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:42.304231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76278523-a092-42c7-9ace-2ae739fedff2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:42.305323 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76278523-a092-42c7-9ace-2ae739fedff2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:42.307475 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76278523-a092-42c7-9ace-2ae739fedff2 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:42.322860 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 254/511] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:08:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:42.324828 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533f59b-4999-4516-9d23-b1c86071f5c8 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:42.325317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533f59b-4999-4516-9d23-b1c86071f5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:42.325784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533f59b-4999-4516-9d23-b1c86071f5c8 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:42.328070 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 255/512] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:42.329304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533f59b-4999-4516-9d23-b1c86071f5c8 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:42.393583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e350f7-b3d5-4894-9655-ff5379e06ff6 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:42.430969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:42.439944 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d2081-9940-496f-8c65-31513355bf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:42.441304 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d2081-9940-496f-8c65-31513355bf79 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:42.441758 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d2081-9940-496f-8c65-31513355bf79 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:42.442171 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d2081-9940-496f-8c65-31513355bf79 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:42.443056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d2081-9940-496f-8c65-31513355bf79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:42.443503 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d2081-9940-496f-8c65-31513355bf79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:42.445081 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d2081-9940-496f-8c65-31513355bf79 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:42.457754 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533f59b-4999-4516-9d23-b1c86071f5c8 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:42.458766 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 258/513] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:42.460453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cd4e8-3942-4121-84e4-c33f04e150c3 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:42.460956 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cd4e8-3942-4121-84e4-c33f04e150c3 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:42.461434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cd4e8-3942-4121-84e4-c33f04e150c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:42.481647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-e91cd4e8-3942-4121-84e4-c33f04e150c3 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:42.483341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e91cd4e8-3942-4121-84e4-c33f04e150c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:42.495590 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3cfa5-3029-4a75-b5fc-f2fe67039117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:42.496544 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3cfa5-3029-4a75-b5fc-f2fe67039117 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:42.496827 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3cfa5-3029-4a75-b5fc-f2fe67039117 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:42.497099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3cfa5-3029-4a75-b5fc-f2fe67039117 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:42.497816 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3cfa5-3029-4a75-b5fc-f2fe67039117 None admin] RBAC: Adding query filter params (role.id=b0f9a090148d4a0eb6456472edec3d5a, scope.project.id=8b2c33250ef84d79a4a4742fb4252156, user.id=7ce720c8df014f9a8af062138aecd1b5) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:42.498121 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3cfa5-3029-4a75-b5fc-f2fe67039117 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:42.500202 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3cfa5-3029-4a75-b5fc-f2fe67039117 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:42.507205 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 256/514] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:08:42 2022] GET /identity/v3/role_assignments?user.id=7ce720c8df014f9a8af062138aecd1b5&scope.project.id=8b2c33250ef84d79a4a4742fb4252156&role.id=b0f9a090148d4a0eb6456472edec3d5a => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:42.508171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3cfa5-3029-4a75-b5fc-f2fe67039117 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:42.524109 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:42.571701 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 259/515] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:42.572535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09949912-f646-4ea5-94e5-ba69b28c0570 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:42.572825 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09949912-f646-4ea5-94e5-ba69b28c0570 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:42.573098 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09949912-f646-4ea5-94e5-ba69b28c0570 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:42.574753 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 260/516] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:42.575547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09949912-f646-4ea5-94e5-ba69b28c0570 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:42.629352 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3cfa5-3029-4a75-b5fc-f2fe67039117 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:42.669357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924a946-0fbc-4fcb-be54-fe6f90d738e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:42.670287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924a946-0fbc-4fcb-be54-fe6f90d738e6 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:42.670712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924a946-0fbc-4fcb-be54-fe6f90d738e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:42.671143 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924a946-0fbc-4fcb-be54-fe6f90d738e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:42.680317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924a946-0fbc-4fcb-be54-fe6f90d738e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:42.682042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924a946-0fbc-4fcb-be54-fe6f90d738e6 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:42.694749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-1924a946-0fbc-4fcb-be54-fe6f90d738e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:42.694749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:42.694749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:42.694749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:42.694749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:42.694749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:42.694749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:42.694749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:42.694749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:42.694749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:42.694749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:42.694749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:42.694749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:08:42.694749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:08:42.694749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:08:42.695923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:08:42.695923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:42.695923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:42.695923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:42.695923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:42.695923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:42.695923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:42.695923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:42.695923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:42.695923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:42.695923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:42.695923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:42.695923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:42.695923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:42.695923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:42.695923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:42.695923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:42.697003 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:08:42.697003 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:08:42.697003 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:08:42.697003 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:08:42.697003 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:08:42.697003 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:08:42.697003 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:42.697003 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:42.699699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 257/517] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:42.701042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa005f1a-d986-43b1-8a2e-1270b8bf7ee9 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:42.701327 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa005f1a-d986-43b1-8a2e-1270b8bf7ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:42.701601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa005f1a-d986-43b1-8a2e-1270b8bf7ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:42.711337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09949912-f646-4ea5-94e5-ba69b28c0570 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:42.728085 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-aa005f1a-d986-43b1-8a2e-1270b8bf7ee9 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:42.730428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-aa005f1a-d986-43b1-8a2e-1270b8bf7ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:42.743884 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:42.759853 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e98515-ae02-4661-85d9-8efa5b06a5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:42.761118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e98515-ae02-4661-85d9-8efa5b06a5c9 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:42.761118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e98515-ae02-4661-85d9-8efa5b06a5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:42.761348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e98515-ae02-4661-85d9-8efa5b06a5c9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:42.762427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e98515-ae02-4661-85d9-8efa5b06a5c9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:42.763453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e98515-ae02-4661-85d9-8efa5b06a5c9 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:42.774790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-e8e98515-ae02-4661-85d9-8efa5b06a5c9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 21:08:42.774790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:42.774790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:08:42.774790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:08:42.774790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:08:42.774790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:08:42.774790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:08:42.774790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:42.774790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 21:08:42.774790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:42.774790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:08:42.774790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:42.774790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:42.774790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:42.774790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:42.774790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:42.774790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:42.776111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:42.776111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:42.776111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:42.776111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:42.776111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:42.776111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:42.776111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:08:42.776111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:08:42.776111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:08:42.776111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:08:42.776111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:42.776111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:42.776111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:42.776111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:42.776111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:42.776111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:42.777152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:42.777152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:42.777152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:42.777152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:42.777152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:42.777152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:42.777152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:42.777152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:42.777152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:42.777152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:42.777152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:08:42.777152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:42.777152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 21:08:42.777152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:42.784087 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 261/518] 10.222.0.13 () {56 vars in 1129 bytes} [Fri Oct 21 21:08:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:42.784087 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e98515-ae02-4661-85d9-8efa5b06a5c9 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:42.819026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 258/519] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:42.820059 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa005f1a-d986-43b1-8a2e-1270b8bf7ee9 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:42.822418 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:42.922639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e98515-ae02-4661-85d9-8efa5b06a5c9 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:42.941652 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa005f1a-d986-43b1-8a2e-1270b8bf7ee9 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:42.969349 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a37d09-b5db-4664-841c-fe8d6736d7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:42.970804 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a37d09-b5db-4664-841c-fe8d6736d7fb None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:42.971232 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a37d09-b5db-4664-841c-fe8d6736d7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:42.971734 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a37d09-b5db-4664-841c-fe8d6736d7fb None admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:42.972807 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a37d09-b5db-4664-841c-fe8d6736d7fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:42.973630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a37d09-b5db-4664-841c-fe8d6736d7fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:42.975509 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a37d09-b5db-4664-841c-fe8d6736d7fb None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:42.978108 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05c5d5-ec0b-4ba7-aec4-0a7a22919f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:42.978858 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05c5d5-ec0b-4ba7-aec4-0a7a22919f2b None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:42.979245 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05c5d5-ec0b-4ba7-aec4-0a7a22919f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:42.979654 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05c5d5-ec0b-4ba7-aec4-0a7a22919f2b None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:42.980388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05c5d5-ec0b-4ba7-aec4-0a7a22919f2b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:42.980802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05c5d5-ec0b-4ba7-aec4-0a7a22919f2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:42.982383 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05c5d5-ec0b-4ba7-aec4-0a7a22919f2b None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:42.989903 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 259/520] 10.222.0.13 () {56 vars in 1124 bytes} [Fri Oct 21 21:08:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:42.990807 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05c5d5-ec0b-4ba7-aec4-0a7a22919f2b None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:42.991290 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 262/521] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:08:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:43.000154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a37d09-b5db-4664-841c-fe8d6736d7fb None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:43.047965 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:43.085735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:43.106425 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a37d09-b5db-4664-841c-fe8d6736d7fb None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:43.126624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05c5d5-ec0b-4ba7-aec4-0a7a22919f2b None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:43.142763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae99b2-c7e4-4f40-972d-3153896db39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:43.143479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae99b2-c7e4-4f40-972d-3153896db39e None admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:43.143788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae99b2-c7e4-4f40-972d-3153896db39e None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:43.144065 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae99b2-c7e4-4f40-972d-3153896db39e None admin] PATH_INFO: `/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/12dad9f016a740b98538722e1b723823/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:43.145955 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae99b2-c7e4-4f40-972d-3153896db39e None admin] RBAC: Authorizing `identity:create_grant(project_id=8b2c33250ef84d79a4a4742fb4252156, user_id=12dad9f016a740b98538722e1b723823, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:43.147399 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae99b2-c7e4-4f40-972d-3153896db39e None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:43.160921 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 263/522] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:08:42 2022] PUT /identity/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/12dad9f016a740b98538722e1b723823/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:08:43.161728 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae99b2-c7e4-4f40-972d-3153896db39e None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:43.175079 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03c99c-fcc4-4109-9f50-4af09477e839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:43.176463 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03c99c-fcc4-4109-9f50-4af09477e839 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:43.176968 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03c99c-fcc4-4109-9f50-4af09477e839 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:43.177755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03c99c-fcc4-4109-9f50-4af09477e839 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:43.195891 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03c99c-fcc4-4109-9f50-4af09477e839 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:43.198744 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03c99c-fcc4-4109-9f50-4af09477e839 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:43.221130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-ee03c99c-fcc4-4109-9f50-4af09477e839 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:43.221130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:43.221130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:43.221130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:43.221130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:43.221130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:43.221130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:43.221130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:43.221130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:43.221130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:43.221130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:43.221130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:43.221130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:43.221130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:43.221130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:43.221130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:43.222193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:43.222193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:43.222193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:43.222193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:43.222193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:43.222193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:43.222193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:43.222193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:43.222193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:43.222193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:43.222193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:43.222193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:43.222193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:43.222193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:43.222193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:43.222193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:43.222193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:43.223047 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:43.223047 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:43.223047 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:43.230869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 260/523] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:43.230869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03c99c-fcc4-4109-9f50-4af09477e839 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:43.286328 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae99b2-c7e4-4f40-972d-3153896db39e None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:43.295510 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:43.328810 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbe2d2-78af-4322-a8c0-127c652c7920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:43.329447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbe2d2-78af-4322-a8c0-127c652c7920 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:43.329946 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbe2d2-78af-4322-a8c0-127c652c7920 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:43.330055 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbe2d2-78af-4322-a8c0-127c652c7920 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:43.330645 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbe2d2-78af-4322-a8c0-127c652c7920 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:43.331019 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbe2d2-78af-4322-a8c0-127c652c7920 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:43.332460 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbe2d2-78af-4322-a8c0-127c652c7920 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:43.339752 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 264/524] 10.222.0.13 () {56 vars in 1124 bytes} [Fri Oct 21 21:08:43 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:43.340634 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbe2d2-78af-4322-a8c0-127c652c7920 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:43.387266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03c99c-fcc4-4109-9f50-4af09477e839 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:43.413043 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:43.452937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb2b76-481d-46bf-bc73-f12de20cd4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:43.453594 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb2b76-481d-46bf-bc73-f12de20cd4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:43.453876 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb2b76-481d-46bf-bc73-f12de20cd4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:43.454216 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb2b76-481d-46bf-bc73-f12de20cd4e9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:43.454809 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb2b76-481d-46bf-bc73-f12de20cd4e9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:43.457945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb2b76-481d-46bf-bc73-f12de20cd4e9 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:43.477772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-9bbb2b76-481d-46bf-bc73-f12de20cd4e9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 21:08:43.477772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:43.477772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:08:43.477772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:08:43.477772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:08:43.477772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:08:43.477772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:08:43.477772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:43.477772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 21:08:43.477772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:43.477772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:08:43.477772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:43.477772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:43.477772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:43.477772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:43.477772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:43.477772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:43.478817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:43.478817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:43.478817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:43.478817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:43.478817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:43.478817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:43.478817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:08:43.478817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:08:43.478817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:08:43.478817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:08:43.478817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:43.478817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:43.478817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:43.478817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:43.478817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:43.478817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:43.479655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:43.479655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:43.479655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:43.479655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:43.479655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:43.479655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:43.479655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:43.479655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:43.479655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:43.479655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:43.479655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:08:43.479655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:43.479655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 21:08:43.479655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:43.479655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 261/525] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:43 2022] GET /identity/v3/services/network => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:43.488936 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb2b76-481d-46bf-bc73-f12de20cd4e9 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:43.507625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbe2d2-78af-4322-a8c0-127c652c7920 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:43.557701 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc72e19-3f73-4c93-b229-335595e1286c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:43.558413 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc72e19-3f73-4c93-b229-335595e1286c None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:43.558699 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc72e19-3f73-4c93-b229-335595e1286c None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:43.558973 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc72e19-3f73-4c93-b229-335595e1286c None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:43.559744 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc72e19-3f73-4c93-b229-335595e1286c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:43.560050 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc72e19-3f73-4c93-b229-335595e1286c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:43.561432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc72e19-3f73-4c93-b229-335595e1286c None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:43.575971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 265/526] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:43.582217 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc72e19-3f73-4c93-b229-335595e1286c None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:43.635411 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:43.651991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb2b76-481d-46bf-bc73-f12de20cd4e9 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:43.700225 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8f987-291d-4d9a-b3cc-beb1b10e715b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:43.700837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8f987-291d-4d9a-b3cc-beb1b10e715b None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:43.701140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8f987-291d-4d9a-b3cc-beb1b10e715b None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:43.701383 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8f987-291d-4d9a-b3cc-beb1b10e715b None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:43.702014 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8f987-291d-4d9a-b3cc-beb1b10e715b None admin] RBAC: Adding query filter params (name=network) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:43.702317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8f987-291d-4d9a-b3cc-beb1b10e715b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:43.703738 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8f987-291d-4d9a-b3cc-beb1b10e715b None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:43.709058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 262/527] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Oct 21 21:08:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:43.717832 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8f987-291d-4d9a-b3cc-beb1b10e715b None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:43.769604 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc72e19-3f73-4c93-b229-335595e1286c None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:43.832728 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:43.834904 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0f8d7-1b9d-4017-b594-ad198cb72d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:43.835643 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0f8d7-1b9d-4017-b594-ad198cb72d51 None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:43.835920 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0f8d7-1b9d-4017-b594-ad198cb72d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:43.836194 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0f8d7-1b9d-4017-b594-ad198cb72d51 None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:43.836876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a0f8d7-1b9d-4017-b594-ad198cb72d51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:43.838325 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a0f8d7-1b9d-4017-b594-ad198cb72d51 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:43.882338 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 266/528] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:08:43 2022] POST /identity/v3/users => generated 254 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:08:43.916094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8f987-291d-4d9a-b3cc-beb1b10e715b None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:43.967871 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36cd70-fe73-4484-8917-baed901febe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:43.968610 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36cd70-fe73-4484-8917-baed901febe6 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:43.968891 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36cd70-fe73-4484-8917-baed901febe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:43.969161 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36cd70-fe73-4484-8917-baed901febe6 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:43.969806 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36cd70-fe73-4484-8917-baed901febe6 None admin] RBAC: Adding query filter params (type=network) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:43.970105 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36cd70-fe73-4484-8917-baed901febe6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:43.972209 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36cd70-fe73-4484-8917-baed901febe6 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:43.981758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 263/529] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Oct 21 21:08:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:44.882647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed072b8-24f3-44b2-9d1f-44ef62e53284 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:44.883158 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed072b8-24f3-44b2-9d1f-44ef62e53284 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:44.883158 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed072b8-24f3-44b2-9d1f-44ef62e53284 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:44.884799 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 267/530] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:44.890199 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01910592-7424-4b64-be0f-0f39ac19226c None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:44.890597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01910592-7424-4b64-be0f-0f39ac19226c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:44.890781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01910592-7424-4b64-be0f-0f39ac19226c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:44.911445 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-01910592-7424-4b64-be0f-0f39ac19226c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:44.912719 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-01910592-7424-4b64-be0f-0f39ac19226c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:44.993271 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 264/531] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:45.322248 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed072b8-24f3-44b2-9d1f-44ef62e53284 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:45.422550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:45.446508 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed072b8-24f3-44b2-9d1f-44ef62e53284 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:45.492388 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778db713-13f7-4fb3-8d6f-dce221864cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:45.493023 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778db713-13f7-4fb3-8d6f-dce221864cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:45.493296 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778db713-13f7-4fb3-8d6f-dce221864cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:45.493564 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778db713-13f7-4fb3-8d6f-dce221864cd7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:45.512456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778db713-13f7-4fb3-8d6f-dce221864cd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:45.517912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-778db713-13f7-4fb3-8d6f-dce221864cd7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:08:45.517912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:45.517912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:45.517912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:45.517912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:45.517912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:45.517912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:45.517912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:45.517912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:45.517912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:45.517912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:45.517912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:45.517912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:45.517912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:45.517912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:45.517912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:45.518842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:45.518842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:45.518842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:45.518842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:45.518842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:45.518842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:45.518842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:45.518842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:45.518842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:45.518842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:45.518842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:45.518842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:45.518842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:45.518842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:45.518842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:45.518842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:45.520089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:45.520089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:45.520089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:45.520089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:45.520089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:45.520089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:45.520089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:45.520089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:45.520089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:45.520089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:45.520089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:08:45.520089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:45.520089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 268/532] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:08:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:45.525190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01910592-7424-4b64-be0f-0f39ac19226c None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:45.589390 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:45.656733 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01910592-7424-4b64-be0f-0f39ac19226c None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:45.693040 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d554572-8174-4afc-851e-c5122fb191d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:45.693645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d554572-8174-4afc-851e-c5122fb191d2 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:45.693919 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d554572-8174-4afc-851e-c5122fb191d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:45.694187 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d554572-8174-4afc-851e-c5122fb191d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:45.694839 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d554572-8174-4afc-851e-c5122fb191d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:45.695140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d554572-8174-4afc-851e-c5122fb191d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:45.696519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d554572-8174-4afc-851e-c5122fb191d2 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:45.705079 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 265/533] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:45.710543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778db713-13f7-4fb3-8d6f-dce221864cd7 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:45.765788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936378af-8ba3-4ff0-8407-06ad8b1a6034 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:45.766117 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936378af-8ba3-4ff0-8407-06ad8b1a6034 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:45.766392 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936378af-8ba3-4ff0-8407-06ad8b1a6034 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:45.767973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 266/534] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:08:45.776549 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adb5ec-34ea-4179-8b49-6b0b6ab41af9 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:45.776830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adb5ec-34ea-4179-8b49-6b0b6ab41af9 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:45.777102 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adb5ec-34ea-4179-8b49-6b0b6ab41af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:45.797217 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-b8adb5ec-34ea-4179-8b49-6b0b6ab41af9 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:45.798683 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-b8adb5ec-34ea-4179-8b49-6b0b6ab41af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:45.874179 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 267/535] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 21:08:45.901933 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778db713-13f7-4fb3-8d6f-dce221864cd7 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:45.945279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ced218-d87c-4b49-8d92-d195b8e16b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:45.945904 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ced218-d87c-4b49-8d92-d195b8e16b31 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:45.946180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ced218-d87c-4b49-8d92-d195b8e16b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:45.946445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ced218-d87c-4b49-8d92-d195b8e16b31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:45.957694 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ced218-d87c-4b49-8d92-d195b8e16b31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:45.959163 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ced218-d87c-4b49-8d92-d195b8e16b31 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:45.973787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-63ced218-d87c-4b49-8d92-d195b8e16b31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:45.973787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:45.973787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:45.973787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:45.973787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:45.973787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:45.973787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:45.973787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:45.973787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:45.973787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:45.973787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:45.973787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:45.973787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:45.973787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:45.973787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:45.973787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:45.974732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:45.974732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:45.974732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:45.974732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:45.974732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:45.974732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:45.974732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:45.974732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:45.974732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:45.974732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:45.974732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:45.974732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:45.974732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:45.974732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:45.974732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:45.974732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:45.974732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:45.976730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:45.976730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:45.976730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:45.976730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 269/536] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:45.976730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:45.980800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8adb5ec-34ea-4179-8b49-6b0b6ab41af9 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:46.011988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:46.071610 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158fda9-154f-4770-86db-2f738d06a43d None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:46.073722 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158fda9-154f-4770-86db-2f738d06a43d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:46.073722 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158fda9-154f-4770-86db-2f738d06a43d None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:46.075338 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 270/537] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:08:46.079383 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fdfeb-20c8-4411-9bd8-73cc7c4be3e6 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:46.079918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fdfeb-20c8-4411-9bd8-73cc7c4be3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:46.080396 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fdfeb-20c8-4411-9bd8-73cc7c4be3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:46.081708 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8adb5ec-34ea-4179-8b49-6b0b6ab41af9 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:46.099185 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-2b5fdfeb-20c8-4411-9bd8-73cc7c4be3e6 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:46.100732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2b5fdfeb-20c8-4411-9bd8-73cc7c4be3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:46.120180 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7a525-af9e-4437-bf73-2a3729826bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:46.120787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7a525-af9e-4437-bf73-2a3729826bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:46.121063 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7a525-af9e-4437-bf73-2a3729826bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:46.121329 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7a525-af9e-4437-bf73-2a3729826bd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:46.122366 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7a525-af9e-4437-bf73-2a3729826bd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:46.122688 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7a525-af9e-4437-bf73-2a3729826bd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:46.124393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7a525-af9e-4437-bf73-2a3729826bd6 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:46.138958 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 268/538] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:46.145675 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7a525-af9e-4437-bf73-2a3729826bd6 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:46.203641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 271/539] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:46.259179 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:46.268034 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7a525-af9e-4437-bf73-2a3729826bd6 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:46.301878 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4a6df-6439-497d-b1aa-235801316de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:46.302509 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4a6df-6439-497d-b1aa-235801316de7 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:46.302783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4a6df-6439-497d-b1aa-235801316de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:46.303053 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4a6df-6439-497d-b1aa-235801316de7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:46.311027 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4a6df-6439-497d-b1aa-235801316de7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:46.312540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4a6df-6439-497d-b1aa-235801316de7 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:46.323840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-a0f4a6df-6439-497d-b1aa-235801316de7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 21 21:08:46.323840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:46.323840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:46.323840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:46.323840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:46.323840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:46.323840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:46.323840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:46.323840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:46.323840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:46.323840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:46.323840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:46.323840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:08:46.323840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:08:46.323840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:08:46.323840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:46.324784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:46.325718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:46.325718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:46.325718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:08:46.325718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:08:46.325718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:08:46.325718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:08:46.325718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:08:46.325718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:08:46.325718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 21 21:08:46.325718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:46.325718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 269/540] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:08:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:08:46.330913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fdfeb-20c8-4411-9bd8-73cc7c4be3e6 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:46.376295 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4a6df-6439-497d-b1aa-235801316de7 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:46.479186 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fdfeb-20c8-4411-9bd8-73cc7c4be3e6 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:46.524962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4a6df-6439-497d-b1aa-235801316de7 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:46.533537 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:46.550624 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e7d3a-e100-4b4c-88d2-2f4655e44b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:46.551519 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e7d3a-e100-4b4c-88d2-2f4655e44b45 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:46.551819 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e7d3a-e100-4b4c-88d2-2f4655e44b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:46.552091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e7d3a-e100-4b4c-88d2-2f4655e44b45 None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:46.552986 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e7d3a-e100-4b4c-88d2-2f4655e44b45 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:46.553660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e7d3a-e100-4b4c-88d2-2f4655e44b45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:46.555537 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e7d3a-e100-4b4c-88d2-2f4655e44b45 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:46.582636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e0f98-83d0-4c72-8bc9-a68a1505ed1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:46.583293 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e0f98-83d0-4c72-8bc9-a68a1505ed1c None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:46.583593 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e0f98-83d0-4c72-8bc9-a68a1505ed1c None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:46.583865 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e0f98-83d0-4c72-8bc9-a68a1505ed1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:46.601690 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 272/541] 10.222.0.13 () {56 vars in 1135 bytes} [Fri Oct 21 21:08:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:46.602741 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e7d3a-e100-4b4c-88d2-2f4655e44b45 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:46.608160 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58e0f98-83d0-4c72-8bc9-a68a1505ed1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:46.619836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-a58e0f98-83d0-4c72-8bc9-a68a1505ed1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:08:46.619836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:46.619836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:46.619836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:46.619836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:46.619836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:46.619836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:46.619836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:46.619836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:46.619836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:46.619836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:46.619836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:46.619836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:46.619836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:46.619836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:46.619836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:46.620846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:46.620846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:46.620846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:46.620846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:46.620846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:46.620846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:46.620846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:46.620846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:46.620846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:46.620846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:46.620846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:46.620846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:46.620846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:46.620846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:46.620846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:46.620846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:46.621678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:46.621678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:46.621678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:46.621678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:46.621678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:46.621678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:46.621678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:46.621678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:46.621678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:46.621678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:46.621678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:08:46.621678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:46.622313 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 270/542] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:08:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:46.622912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e0f98-83d0-4c72-8bc9-a68a1505ed1c None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:46.652012 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:46.713289 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e7d3a-e100-4b4c-88d2-2f4655e44b45 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:46.739525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e0f98-83d0-4c72-8bc9-a68a1505ed1c None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:46.753990 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6763f-8d9f-4a52-a270-4034e6601b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:46.754618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6763f-8d9f-4a52-a270-4034e6601b28 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:46.754895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6763f-8d9f-4a52-a270-4034e6601b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:46.755165 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6763f-8d9f-4a52-a270-4034e6601b28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:46.766284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6763f-8d9f-4a52-a270-4034e6601b28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:46.767819 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6763f-8d9f-4a52-a270-4034e6601b28 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:46.775906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52320a89-6c45-4a65-b958-3349e2836e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:46.776601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52320a89-6c45-4a65-b958-3349e2836e5c None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:46.776970 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52320a89-6c45-4a65-b958-3349e2836e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:46.777255 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52320a89-6c45-4a65-b958-3349e2836e5c None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:46.777901 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52320a89-6c45-4a65-b958-3349e2836e5c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:46.779258 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52320a89-6c45-4a65-b958-3349e2836e5c None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:46.785901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-c8d6763f-8d9f-4a52-a270-4034e6601b28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:46.785901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:46.785901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:46.785901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:46.785901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:46.785901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:46.785901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:46.785901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:46.785901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:46.785901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:46.785901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:46.785901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:46.785901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:46.785901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:46.785901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:46.785901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:46.786832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:46.786832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:46.786832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:46.786832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:46.786832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:46.786832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:46.786832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:46.786832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:46.786832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:46.786832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:46.786832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:46.786832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:46.786832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:46.786832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:46.786832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:46.786832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:46.786832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:46.787716 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:46.787716 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:46.787716 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:46.793647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 273/543] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:46.793647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6763f-8d9f-4a52-a270-4034e6601b28 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:46.802239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 271/544] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Oct 21 21:08:46 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:08:46.802239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45554487-06ac-4b91-816a-faeee23f05f6 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:46.802239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45554487-06ac-4b91-816a-faeee23f05f6 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:46.802239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45554487-06ac-4b91-816a-faeee23f05f6 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:46.805281 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 272/545] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:46.806231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45554487-06ac-4b91-816a-faeee23f05f6 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:46.850565 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:46.899293 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:46.944267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6763f-8d9f-4a52-a270-4034e6601b28 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:46.958314 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45554487-06ac-4b91-816a-faeee23f05f6 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:46.988188 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79ffcb-ef83-424d-b749-9088089f4254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:46.989494 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79ffcb-ef83-424d-b749-9088089f4254 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:46.989494 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79ffcb-ef83-424d-b749-9088089f4254 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:46.989494 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79ffcb-ef83-424d-b749-9088089f4254 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:46.989975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79ffcb-ef83-424d-b749-9088089f4254 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:46.990273 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79ffcb-ef83-424d-b749-9088089f4254 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:46.991711 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79ffcb-ef83-424d-b749-9088089f4254 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:47.000486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8bfe4-cfbc-48c1-8b52-274de96fe55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:47.000973 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 274/546] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:47.001733 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8bfe4-cfbc-48c1-8b52-274de96fe55f None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:47.002174 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8bfe4-cfbc-48c1-8b52-274de96fe55f None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:47.002445 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8bfe4-cfbc-48c1-8b52-274de96fe55f None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:47.003113 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8bfe4-cfbc-48c1-8b52-274de96fe55f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:47.003414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8bfe4-cfbc-48c1-8b52-274de96fe55f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:47.004797 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8bfe4-cfbc-48c1-8b52-274de96fe55f None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:47.004897 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cedbdd-99dd-49d1-9e94-71cc20072f36 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:47.005153 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cedbdd-99dd-49d1-9e94-71cc20072f36 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:47.005565 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cedbdd-99dd-49d1-9e94-71cc20072f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:47.018882 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 273/547] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:47.022929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8cc84-4094-4346-bfe0-4a874b57d2cd None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:47.022929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8cc84-4094-4346-bfe0-4a874b57d2cd None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:47.022929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8cc84-4094-4346-bfe0-4a874b57d2cd None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:47.022929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 274/548] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:47.022929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd8cc84-4094-4346-bfe0-4a874b57d2cd None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:47.028172 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-67cedbdd-99dd-49d1-9e94-71cc20072f36 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:47.029389 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-67cedbdd-99dd-49d1-9e94-71cc20072f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:47.103193 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 275/549] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:47.103896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cedbdd-99dd-49d1-9e94-71cc20072f36 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:47.136364 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:47.145813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd8cc84-4094-4346-bfe0-4a874b57d2cd None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:47.181901 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:47.194413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c843867-6e2e-4c6c-bd62-f316544060b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:47.195342 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c843867-6e2e-4c6c-bd62-f316544060b6 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:47.196289 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c843867-6e2e-4c6c-bd62-f316544060b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:47.196772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c843867-6e2e-4c6c-bd62-f316544060b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:47.208009 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cedbdd-99dd-49d1-9e94-71cc20072f36 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:47.211511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c843867-6e2e-4c6c-bd62-f316544060b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:47.213243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c843867-6e2e-4c6c-bd62-f316544060b6 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:47.234734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-0c843867-6e2e-4c6c-bd62-f316544060b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:47.234734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:47.234734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:47.234734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:47.234734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:47.234734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:47.234734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:47.234734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:47.234734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:47.234734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:47.234734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:47.234734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:47.234734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:47.234734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:47.234734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:47.234734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:47.235829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:47.235829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:47.235829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:47.235829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:47.235829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:47.235829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:47.235829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:47.235829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:47.235829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:47.235829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:47.235829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:47.235829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:47.235829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:47.235829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:47.235829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:47.235829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:47.235829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:47.236685 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:47.236685 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:47.236685 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:47.238802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 275/550] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:47.240306 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0f978-eb7d-4ef7-834f-c0be45a60730 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:47.241036 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0f978-eb7d-4ef7-834f-c0be45a60730 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:47.241715 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0f978-eb7d-4ef7-834f-c0be45a60730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:47.245835 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4261519-1b41-4ebd-ad25-7bfc036c68d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:47.246594 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4261519-1b41-4ebd-ad25-7bfc036c68d6 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:47.246975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4261519-1b41-4ebd-ad25-7bfc036c68d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:47.247351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4261519-1b41-4ebd-ad25-7bfc036c68d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:47.254721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4261519-1b41-4ebd-ad25-7bfc036c68d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:47.256357 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4261519-1b41-4ebd-ad25-7bfc036c68d6 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:47.263032 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-80b0f978-eb7d-4ef7-834f-c0be45a60730 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:47.264651 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-80b0f978-eb7d-4ef7-834f-c0be45a60730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:47.269447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-a4261519-1b41-4ebd-ad25-7bfc036c68d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:47.269447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:47.269447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:47.269447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:47.269447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:47.269447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:47.269447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:47.269447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:47.269447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:47.269447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:47.269447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:47.269447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:47.269447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:08:47.269447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:08:47.269447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:08:47.270487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:08:47.270487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:47.270487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:47.270487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:47.270487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:47.270487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:47.270487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:47.270487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:47.270487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:47.270487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:47.270487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:47.270487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:47.270487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:47.270487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:47.270487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:47.270487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:47.270487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:47.271557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:08:47.271557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:08:47.271557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:08:47.271557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:08:47.271557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:08:47.271557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:08:47.271557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:47.271557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:47.278006 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 276/551] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:47.279032 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4261519-1b41-4ebd-ad25-7bfc036c68d6 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:47.355656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 276/552] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 21:08:47.356868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0f978-eb7d-4ef7-834f-c0be45a60730 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:47.408919 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:47.422821 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4261519-1b41-4ebd-ad25-7bfc036c68d6 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:47.468780 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca7628-ef42-44a3-9f2f-0c81b578dfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:47.469696 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca7628-ef42-44a3-9f2f-0c81b578dfa0 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:47.469991 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca7628-ef42-44a3-9f2f-0c81b578dfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:47.470269 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca7628-ef42-44a3-9f2f-0c81b578dfa0 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:47.470916 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca7628-ef42-44a3-9f2f-0c81b578dfa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:47.471244 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca7628-ef42-44a3-9f2f-0c81b578dfa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:47.472687 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca7628-ef42-44a3-9f2f-0c81b578dfa0 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:47.488632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0f978-eb7d-4ef7-834f-c0be45a60730 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:47.488799 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 277/553] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:47.489531 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca7628-ef42-44a3-9f2f-0c81b578dfa0 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:47.501514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:47.528419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc99248-177c-47aa-ab20-e29ccc07ed61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:47.529417 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc99248-177c-47aa-ab20-e29ccc07ed61 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:47.529899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc99248-177c-47aa-ab20-e29ccc07ed61 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:47.530370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc99248-177c-47aa-ab20-e29ccc07ed61 None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:47.531250 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc99248-177c-47aa-ab20-e29ccc07ed61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:47.531767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc99248-177c-47aa-ab20-e29ccc07ed61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:47.533392 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc99248-177c-47aa-ab20-e29ccc07ed61 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:47.545920 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 277/554] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:08:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:47.547051 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc99248-177c-47aa-ab20-e29ccc07ed61 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:47.631130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca7628-ef42-44a3-9f2f-0c81b578dfa0 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:47.653416 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc99248-177c-47aa-ab20-e29ccc07ed61 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:47.675647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef5510-034c-468b-9b57-4335d7e281e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:47.676360 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef5510-034c-468b-9b57-4335d7e281e3 None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:47.676712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef5510-034c-468b-9b57-4335d7e281e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:47.677061 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef5510-034c-468b-9b57-4335d7e281e3 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:47.677775 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef5510-034c-468b-9b57-4335d7e281e3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:47.679259 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef5510-034c-468b-9b57-4335d7e281e3 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:47.687930 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba640d-ae5e-4421-8ed5-b3c94ddc9461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:47.688670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba640d-ae5e-4421-8ed5-b3c94ddc9461 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:47.688953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba640d-ae5e-4421-8ed5-b3c94ddc9461 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:47.689196 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba640d-ae5e-4421-8ed5-b3c94ddc9461 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:47.696015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 278/555] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Oct 21 21:08:47 2022] POST /identity/v3/services => generated 240 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:08:47.697424 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba640d-ae5e-4421-8ed5-b3c94ddc9461 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:47.698884 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba640d-ae5e-4421-8ed5-b3c94ddc9461 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:47.704602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef5510-034c-468b-9b57-4335d7e281e3 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:47.714962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-17ba640d-ae5e-4421-8ed5-b3c94ddc9461 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 21 21:08:47.714962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:47.714962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:47.714962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:47.714962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:47.714962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:47.714962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:47.714962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:47.714962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:47.714962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:47.714962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:47.714962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:47.714962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:08:47.714962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:08:47.714962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:08:47.714962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:47.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:47.716950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:47.716950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:47.716950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:08:47.716950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:08:47.716950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:08:47.716950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:08:47.716950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:08:47.716950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:08:47.716950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 21 21:08:47.716950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:47.716950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 278/556] 10.222.0.13 () {56 vars in 1093 bytes} [Fri Oct 21 21:08:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:47.717555 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba640d-ae5e-4421-8ed5-b3c94ddc9461 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:47.728542 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:47.760564 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:47.847817 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef5510-034c-468b-9b57-4335d7e281e3 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:47.867468 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba640d-ae5e-4421-8ed5-b3c94ddc9461 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:47.888932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89813adf-46d3-4f2a-939a-2541a0d4f41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:47.889688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89813adf-46d3-4f2a-939a-2541a0d4f41e None admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:47.891170 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89813adf-46d3-4f2a-939a-2541a0d4f41e None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:47.891170 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89813adf-46d3-4f2a-939a-2541a0d4f41e None admin] PATH_INFO: `/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/7ce720c8df014f9a8af062138aecd1b5/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:47.892658 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89813adf-46d3-4f2a-939a-2541a0d4f41e None admin] RBAC: Authorizing `identity:create_grant(project_id=8b2c33250ef84d79a4a4742fb4252156, user_id=7ce720c8df014f9a8af062138aecd1b5, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:47.894196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89813adf-46d3-4f2a-939a-2541a0d4f41e None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:47.909265 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 279/557] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:08:47 2022] PUT /identity/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/7ce720c8df014f9a8af062138aecd1b5/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:08:47.910069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89813adf-46d3-4f2a-939a-2541a0d4f41e None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:47.921343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3929b-c299-46ae-b92e-7d1d70031532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:47.923238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3929b-c299-46ae-b92e-7d1d70031532 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:47.923858 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3929b-c299-46ae-b92e-7d1d70031532 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:47.924411 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3929b-c299-46ae-b92e-7d1d70031532 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:47.941721 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad3929b-c299-46ae-b92e-7d1d70031532 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:47.951628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-9ad3929b-c299-46ae-b92e-7d1d70031532 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:47.951628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:47.951628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:47.951628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:47.951628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:47.951628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:47.951628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:47.951628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:47.951628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:47.951628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:47.951628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:47.951628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:47.951628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:47.951628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:47.951628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:47.951628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:47.952645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:47.952645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:47.952645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:47.952645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:47.952645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:47.952645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:47.952645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:47.952645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:47.952645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:47.952645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:47.952645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:47.952645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:47.952645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:47.952645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:47.952645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:47.952645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:47.953458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:47.953458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:47.953458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:47.953458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:47.953458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:47.953458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:47.953458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:47.953458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:47.953458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:47.953458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:47.953458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:47.953458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:47.956835 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 279/558] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Oct 21 21:08:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:47.962698 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3929b-c299-46ae-b92e-7d1d70031532 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:48.001958 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:48.056059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89813adf-46d3-4f2a-939a-2541a0d4f41e None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:48.072981 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:48.086591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3929b-c299-46ae-b92e-7d1d70031532 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:48.115487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5babb-7d84-4b0f-978f-bfba24da4277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:48.116139 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5babb-7d84-4b0f-978f-bfba24da4277 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:48.116423 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5babb-7d84-4b0f-978f-bfba24da4277 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:48.116701 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5babb-7d84-4b0f-978f-bfba24da4277 None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:48.117427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5babb-7d84-4b0f-978f-bfba24da4277 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:48.117736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5babb-7d84-4b0f-978f-bfba24da4277 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:48.119672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5babb-7d84-4b0f-978f-bfba24da4277 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:48.135849 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36707c54-880f-46ce-8e69-e25adaa86abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:48.136805 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36707c54-880f-46ce-8e69-e25adaa86abf None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:48.137380 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36707c54-880f-46ce-8e69-e25adaa86abf None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:48.137870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36707c54-880f-46ce-8e69-e25adaa86abf None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:48.138993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36707c54-880f-46ce-8e69-e25adaa86abf None admin] RBAC: Adding query filter params (name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:48.139503 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36707c54-880f-46ce-8e69-e25adaa86abf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:48.141074 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36707c54-880f-46ce-8e69-e25adaa86abf None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:48.141570 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 280/559] 10.222.0.13 () {56 vars in 1133 bytes} [Fri Oct 21 21:08:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:48.146995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5babb-7d84-4b0f-978f-bfba24da4277 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:48.153107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 280/560] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Oct 21 21:08:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:48.158971 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36707c54-880f-46ce-8e69-e25adaa86abf None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:48.276528 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5babb-7d84-4b0f-978f-bfba24da4277 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:48.304105 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:48.316568 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36707c54-880f-46ce-8e69-e25adaa86abf None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:48.317181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d06dc-49bb-40cb-9c4d-f22809c5f393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:48.318169 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d06dc-49bb-40cb-9c4d-f22809c5f393 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:48.318655 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d06dc-49bb-40cb-9c4d-f22809c5f393 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:48.319125 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d06dc-49bb-40cb-9c4d-f22809c5f393 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:48.396059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d06dc-49bb-40cb-9c4d-f22809c5f393 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:48.397937 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d06dc-49bb-40cb-9c4d-f22809c5f393 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:48.417870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267da291-ff90-49d8-a677-a6136f4215fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:48.418524 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267da291-ff90-49d8-a677-a6136f4215fe None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:48.418803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267da291-ff90-49d8-a677-a6136f4215fe None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:48.419073 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267da291-ff90-49d8-a677-a6136f4215fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:48.443371 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267da291-ff90-49d8-a677-a6136f4215fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:48.444866 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267da291-ff90-49d8-a677-a6136f4215fe None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:48.469575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-267da291-ff90-49d8-a677-a6136f4215fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:48.469575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:48.469575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:48.469575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:48.469575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:48.469575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:48.469575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:48.469575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:48.469575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:48.469575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:48.469575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:48.469575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:48.469575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:48.469575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:48.469575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:48.469575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:48.470578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:48.470578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:48.470578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:48.470578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:48.470578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:48.470578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:48.470578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:48.470578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:48.470578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:48.470578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:48.470578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:48.470578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:48.470578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:48.470578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:48.470578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:48.470578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:48.470578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:48.471991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:48.471991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:48.471991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:48.471991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 281/561] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:48.476921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267da291-ff90-49d8-a677-a6136f4215fe None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:48.488117 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:48.519406 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 281/562] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:08:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 21:08:48.525132 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d06dc-49bb-40cb-9c4d-f22809c5f393 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:48.592692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:48.649879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267da291-ff90-49d8-a677-a6136f4215fe None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:48.695112 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d06dc-49bb-40cb-9c4d-f22809c5f393 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:48.698812 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86f139-c575-40f3-a0dd-128c027f647a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:48.699425 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86f139-c575-40f3-a0dd-128c027f647a None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:48.699737 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86f139-c575-40f3-a0dd-128c027f647a None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:48.700009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86f139-c575-40f3-a0dd-128c027f647a None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:48.700651 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86f139-c575-40f3-a0dd-128c027f647a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:48.700952 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86f139-c575-40f3-a0dd-128c027f647a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:48.702337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86f139-c575-40f3-a0dd-128c027f647a None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:48.714905 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 282/563] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:48.720533 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86f139-c575-40f3-a0dd-128c027f647a None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:48.745370 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd75ad4-34aa-4dd8-965f-353d0b6eac1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:48.746075 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd75ad4-34aa-4dd8-965f-353d0b6eac1d None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:48.746362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd75ad4-34aa-4dd8-965f-353d0b6eac1d None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:48.746638 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd75ad4-34aa-4dd8-965f-353d0b6eac1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:48.761128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd75ad4-34aa-4dd8-965f-353d0b6eac1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:48.767675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd75ad4-34aa-4dd8-965f-353d0b6eac1d None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:48.799376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-2fd75ad4-34aa-4dd8-965f-353d0b6eac1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:48.799376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:48.799376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:48.799376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:48.799376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:48.799376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:48.799376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:48.799376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:48.799376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:48.799376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:48.799376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:48.799376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:48.799376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:48.799376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:48.799376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:48.799376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:48.800395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:48.800395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:48.800395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:48.800395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:48.800395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:48.800395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:48.800395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:48.800395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:48.800395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:48.800395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:48.800395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:48.800395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:48.800395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:48.800395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:48.800395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:48.800395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:48.800395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:48.801337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:48.801337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:48.801337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:48.807751 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 282/564] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:48.808674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd75ad4-34aa-4dd8-965f-353d0b6eac1d None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:48.821689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:48.845000 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86f139-c575-40f3-a0dd-128c027f647a None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:48.880983 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ba62d-24a7-425a-9463-79eb3e98d018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:48.882092 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ba62d-24a7-425a-9463-79eb3e98d018 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:48.882455 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ba62d-24a7-425a-9463-79eb3e98d018 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:48.882792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ba62d-24a7-425a-9463-79eb3e98d018 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:48.892378 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ba62d-24a7-425a-9463-79eb3e98d018 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:48.894657 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ba62d-24a7-425a-9463-79eb3e98d018 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:48.908659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-3f9ba62d-24a7-425a-9463-79eb3e98d018 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 21 21:08:48.908659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:48.908659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:48.908659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:48.908659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:48.908659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:48.908659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:48.908659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:48.908659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:48.908659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:48.908659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:48.908659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:48.908659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:08:48.908659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:08:48.908659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:08:48.908659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:48.909711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:48.910676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:48.910676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:48.910676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:08:48.910676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:08:48.910676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:08:48.910676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:08:48.910676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:08:48.910676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:08:48.910676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 21 21:08:48.910676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:48.910676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 283/565] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Oct 21 21:08:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:48.915455 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ba62d-24a7-425a-9463-79eb3e98d018 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:49.027967 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ba62d-24a7-425a-9463-79eb3e98d018 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:49.056567 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:49.065561 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a635d28-5c7f-40dc-b9e5-81d856cf02b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:49.066561 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a635d28-5c7f-40dc-b9e5-81d856cf02b5 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:49.067019 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a635d28-5c7f-40dc-b9e5-81d856cf02b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:49.067470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a635d28-5c7f-40dc-b9e5-81d856cf02b5 None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:49.068316 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a635d28-5c7f-40dc-b9e5-81d856cf02b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:49.068632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a635d28-5c7f-40dc-b9e5-81d856cf02b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:49.070224 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a635d28-5c7f-40dc-b9e5-81d856cf02b5 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:49.090455 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 284/566] 10.222.0.13 () {56 vars in 1149 bytes} [Fri Oct 21 21:08:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:49.097019 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a635d28-5c7f-40dc-b9e5-81d856cf02b5 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:49.101849 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:49.150210 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd75ad4-34aa-4dd8-965f-353d0b6eac1d None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:49.184400 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dff437-fadf-4f04-8b56-223b725800e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:49.185063 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dff437-fadf-4f04-8b56-223b725800e6 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:49.185341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dff437-fadf-4f04-8b56-223b725800e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:49.185612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dff437-fadf-4f04-8b56-223b725800e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:49.186293 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dff437-fadf-4f04-8b56-223b725800e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:49.186598 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dff437-fadf-4f04-8b56-223b725800e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:49.188022 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dff437-fadf-4f04-8b56-223b725800e6 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:49.201443 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 283/567] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:49.202686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702a461-e73d-4f2e-bb97-e840f749ff33 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:49.202972 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702a461-e73d-4f2e-bb97-e840f749ff33 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:49.203250 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702a461-e73d-4f2e-bb97-e840f749ff33 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:49.209277 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 284/568] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:49.210076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702a461-e73d-4f2e-bb97-e840f749ff33 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:49.217044 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a635d28-5c7f-40dc-b9e5-81d856cf02b5 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:49.264273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc94ccb-5a31-4b19-8645-8e2edaefe3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:49.264901 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc94ccb-5a31-4b19-8645-8e2edaefe3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:49.265174 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc94ccb-5a31-4b19-8645-8e2edaefe3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:49.265441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc94ccb-5a31-4b19-8645-8e2edaefe3d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:49.279940 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc94ccb-5a31-4b19-8645-8e2edaefe3d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:49.282252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc94ccb-5a31-4b19-8645-8e2edaefe3d6 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:49.301485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-5cc94ccb-5a31-4b19-8645-8e2edaefe3d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:49.301485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:49.301485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:49.301485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:49.301485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:49.301485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:49.301485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:49.301485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:49.301485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:49.301485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:49.301485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:49.301485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:49.301485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:49.301485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:49.301485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:49.301485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:49.302429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:49.302429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:49.302429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:49.302429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:49.302429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:49.302429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:49.302429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:49.302429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:49.302429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:49.302429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:49.302429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:49.302429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:49.302429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:49.302429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:49.302429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:49.302429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:49.302429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:49.303319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:49.303319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:49.303319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:49.307799 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 285/569] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:49.309413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103891f-b844-4ad7-9228-242797e8dd06 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:49.309696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103891f-b844-4ad7-9228-242797e8dd06 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:49.310402 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103891f-b844-4ad7-9228-242797e8dd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:49.320389 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702a461-e73d-4f2e-bb97-e840f749ff33 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:49.333504 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-9103891f-b844-4ad7-9228-242797e8dd06 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:49.334950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9103891f-b844-4ad7-9228-242797e8dd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:49.357862 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4970f0-1ff4-4dc9-a07a-502ae1be0e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:49.358503 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4970f0-1ff4-4dc9-a07a-502ae1be0e41 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:49.358786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4970f0-1ff4-4dc9-a07a-502ae1be0e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:49.359061 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4970f0-1ff4-4dc9-a07a-502ae1be0e41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:49.362199 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:49.367648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4970f0-1ff4-4dc9-a07a-502ae1be0e41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:49.369394 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4970f0-1ff4-4dc9-a07a-502ae1be0e41 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:49.370176 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:49.383800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-6f4970f0-1ff4-4dc9-a07a-502ae1be0e41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:49.383800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:49.383800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:49.383800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:49.383800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:49.383800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:49.383800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:49.383800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:49.383800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:49.383800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:49.383800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:49.383800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:49.383800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:08:49.383800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:08:49.383800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:08:49.384788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:08:49.384788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:49.384788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:49.384788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:49.384788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:49.384788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:49.384788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:49.384788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:49.384788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:49.384788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:49.384788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:49.384788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:49.384788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:49.384788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:49.384788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:49.384788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:49.384788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:49.385663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:08:49.385663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:08:49.385663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:08:49.385663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:08:49.385663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:08:49.385663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:08:49.385663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:49.385663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:49.388223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 285/570] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:49.389618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4970f0-1ff4-4dc9-a07a-502ae1be0e41 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:49.427593 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 286/571] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:08:49.427593 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103891f-b844-4ad7-9228-242797e8dd06 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:49.537197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103891f-b844-4ad7-9228-242797e8dd06 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:49.556190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4970f0-1ff4-4dc9-a07a-502ae1be0e41 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:49.578469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf22a2-0709-40a3-98d2-37fe92d16f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:49.579074 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf22a2-0709-40a3-98d2-37fe92d16f30 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:49.579349 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf22a2-0709-40a3-98d2-37fe92d16f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:49.579639 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf22a2-0709-40a3-98d2-37fe92d16f30 None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:49.581397 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf22a2-0709-40a3-98d2-37fe92d16f30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:49.581702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf22a2-0709-40a3-98d2-37fe92d16f30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:49.583371 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf22a2-0709-40a3-98d2-37fe92d16f30 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:49.600234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 287/572] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:08:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:49.609633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf22a2-0709-40a3-98d2-37fe92d16f30 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:49.616301 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a26ff3-55db-4faa-b029-e7ab472796cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:49.617188 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a26ff3-55db-4faa-b029-e7ab472796cb None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:49.617722 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a26ff3-55db-4faa-b029-e7ab472796cb None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:49.618249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a26ff3-55db-4faa-b029-e7ab472796cb None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:49.619089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a26ff3-55db-4faa-b029-e7ab472796cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:49.619644 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a26ff3-55db-4faa-b029-e7ab472796cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:49.621236 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a26ff3-55db-4faa-b029-e7ab472796cb None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:49.631835 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:49.640601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 286/573] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:49.652967 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a26ff3-55db-4faa-b029-e7ab472796cb None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:49.720811 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf22a2-0709-40a3-98d2-37fe92d16f30 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:49.770330 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a378c-d055-4232-aff0-aa212f47006c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:49.771207 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a378c-d055-4232-aff0-aa212f47006c None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:49.771647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a378c-d055-4232-aff0-aa212f47006c None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:49.772079 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a378c-d055-4232-aff0-aa212f47006c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:49.772935 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a378c-d055-4232-aff0-aa212f47006c None admin] RBAC: Adding query filter params (role.id=c514b065b7604094b899c586f79b0c5b, scope.project.id=8b2c33250ef84d79a4a4742fb4252156, user.id=12dad9f016a740b98538722e1b723823) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:49.774564 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a378c-d055-4232-aff0-aa212f47006c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:49.777247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a378c-d055-4232-aff0-aa212f47006c None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:49.779989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:49.790869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 288/574] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:08:49 2022] GET /identity/v3/role_assignments?user.id=12dad9f016a740b98538722e1b723823&scope.project.id=8b2c33250ef84d79a4a4742fb4252156&role.id=c514b065b7604094b899c586f79b0c5b => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:49.808137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a378c-d055-4232-aff0-aa212f47006c None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:49.862702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a26ff3-55db-4faa-b029-e7ab472796cb None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:49.924219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:49.926771 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c717ee5-d56b-4055-87db-ff790a246613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:49.927547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c717ee5-d56b-4055-87db-ff790a246613 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:49.927852 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c717ee5-d56b-4055-87db-ff790a246613 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:49.928123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c717ee5-d56b-4055-87db-ff790a246613 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:49.940530 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a378c-d055-4232-aff0-aa212f47006c None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:49.940980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c717ee5-d56b-4055-87db-ff790a246613 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:49.942486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c717ee5-d56b-4055-87db-ff790a246613 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:49.963612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-2c717ee5-d56b-4055-87db-ff790a246613 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:49.963612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:49.963612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:49.963612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:49.963612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:49.963612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:49.963612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:49.963612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:49.963612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:49.963612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:49.963612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:49.963612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:49.963612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:08:49.963612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:08:49.963612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:08:49.964583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:08:49.964583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:49.964583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:49.964583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:49.964583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:49.964583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:49.964583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:49.964583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:49.964583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:49.964583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:49.964583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:49.964583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:49.964583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:49.964583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:49.964583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:49.964583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:49.964583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:49.965442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:08:49.965442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:08:49.965442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:08:49.965442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:08:49.965442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:08:49.965442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:08:49.965442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:49.965442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:49.965442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 287/575] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:08:49.970432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c717ee5-d56b-4055-87db-ff790a246613 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:49.984304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a122fbd-79ed-437f-95b0-5933e646a22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:49.984958 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a122fbd-79ed-437f-95b0-5933e646a22e None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:49.985238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a122fbd-79ed-437f-95b0-5933e646a22e None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:49.985509 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a122fbd-79ed-437f-95b0-5933e646a22e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:49.986098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a122fbd-79ed-437f-95b0-5933e646a22e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:49.989202 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a122fbd-79ed-437f-95b0-5933e646a22e None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:50.006466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-2a122fbd-79ed-437f-95b0-5933e646a22e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 21:08:50.006466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:50.006466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:08:50.006466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:08:50.006466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:08:50.006466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:08:50.006466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:08:50.006466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:50.006466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 21:08:50.006466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:50.006466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:08:50.006466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:50.006466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:50.006466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:50.006466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:50.006466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:50.006466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:50.008964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:50.008964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:50.008964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:50.008964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:50.008964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:50.008964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:50.008964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:08:50.008964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:08:50.008964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:08:50.008964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:08:50.008964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:50.008964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:50.008964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:50.008964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:50.008964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:50.008964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:50.010988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:50.010988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:50.010988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:50.010988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:50.010988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:50.010988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:50.010988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:50.010988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:50.010988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:50.010988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:50.010988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:08:50.010988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:50.010988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 21:08:50.010988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:50.010988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 289/576] 10.222.0.13 () {56 vars in 1129 bytes} [Fri Oct 21 21:08:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:08:50.016094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a122fbd-79ed-437f-95b0-5933e646a22e None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:50.147468 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c717ee5-d56b-4055-87db-ff790a246613 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:50.171360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a122fbd-79ed-437f-95b0-5933e646a22e None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:50.179962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:50.193215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24416dd-1ecf-40a7-aa03-7769897f5243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:50.193838 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24416dd-1ecf-40a7-aa03-7769897f5243 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:50.194141 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24416dd-1ecf-40a7-aa03-7769897f5243 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:50.194409 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24416dd-1ecf-40a7-aa03-7769897f5243 None admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:50.196167 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24416dd-1ecf-40a7-aa03-7769897f5243 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:50.196502 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24416dd-1ecf-40a7-aa03-7769897f5243 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:50.197988 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24416dd-1ecf-40a7-aa03-7769897f5243 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:50.212309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 288/577] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:08:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:50.216316 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf7338-095a-45fe-b698-c9a50d99e391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:50.217252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf7338-095a-45fe-b698-c9a50d99e391 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:50.217540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf7338-095a-45fe-b698-c9a50d99e391 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:50.217812 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf7338-095a-45fe-b698-c9a50d99e391 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:50.217975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24416dd-1ecf-40a7-aa03-7769897f5243 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:50.218438 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf7338-095a-45fe-b698-c9a50d99e391 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:50.218742 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf7338-095a-45fe-b698-c9a50d99e391 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:50.220127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf7338-095a-45fe-b698-c9a50d99e391 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:50.225953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 290/578] 10.222.0.13 () {56 vars in 1124 bytes} [Fri Oct 21 21:08:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:50.233388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf7338-095a-45fe-b698-c9a50d99e391 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:50.259126 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:50.354254 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24416dd-1ecf-40a7-aa03-7769897f5243 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:50.366490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf7338-095a-45fe-b698-c9a50d99e391 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:50.399972 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f09c6-0fdc-47c9-874a-3cf8cec4c6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:50.400680 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f09c6-0fdc-47c9-874a-3cf8cec4c6e9 None admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:50.400962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f09c6-0fdc-47c9-874a-3cf8cec4c6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:50.401392 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f09c6-0fdc-47c9-874a-3cf8cec4c6e9 None admin] PATH_INFO: `/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/2346003afe74477496045e9bc72173d5/roles/b0f9a090148d4a0eb6456472edec3d5a` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:50.408178 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fb22d-c0a5-499d-9864-4c113e3fd2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:50.408860 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fb22d-c0a5-499d-9864-4c113e3fd2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:50.409141 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fb22d-c0a5-499d-9864-4c113e3fd2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:50.409412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fb22d-c0a5-499d-9864-4c113e3fd2e5 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:50.410048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fb22d-c0a5-499d-9864-4c113e3fd2e5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:50.410349 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fb22d-c0a5-499d-9864-4c113e3fd2e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:50.412452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fb22d-c0a5-499d-9864-4c113e3fd2e5 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:50.419395 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 291/579] 10.222.0.13 () {56 vars in 1124 bytes} [Fri Oct 21 21:08:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:50.419719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f09c6-0fdc-47c9-874a-3cf8cec4c6e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b2c33250ef84d79a4a4742fb4252156, user_id=2346003afe74477496045e9bc72173d5, role_id=b0f9a090148d4a0eb6456472edec3d5a)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:50.420813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57812b89-f7a3-48a4-b979-999aabf3e637 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:50.421101 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57812b89-f7a3-48a4-b979-999aabf3e637 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:50.421229 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f09c6-0fdc-47c9-874a-3cf8cec4c6e9 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:50.421426 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57812b89-f7a3-48a4-b979-999aabf3e637 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:50.423656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 292/580] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:50.426694 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57812b89-f7a3-48a4-b979-999aabf3e637 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:50.447580 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 289/581] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:08:50 2022] PUT /identity/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/2346003afe74477496045e9bc72173d5/roles/b0f9a090148d4a0eb6456472edec3d5a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:08:50.449103 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d564fb7-4096-480c-9fbb-b310674d3c35 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:50.449453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d564fb7-4096-480c-9fbb-b310674d3c35 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:50.449730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d564fb7-4096-480c-9fbb-b310674d3c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:50.472283 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-2d564fb7-4096-480c-9fbb-b310674d3c35 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:50.473541 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2d564fb7-4096-480c-9fbb-b310674d3c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:50.500365 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:50.522548 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:50.544630 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57812b89-f7a3-48a4-b979-999aabf3e637 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:50.566203 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 290/582] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:08:50.590145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfc469-1bdf-4147-9bfd-b0c729988449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:50.590760 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfc469-1bdf-4147-9bfd-b0c729988449 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:50.591039 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfc469-1bdf-4147-9bfd-b0c729988449 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:50.591310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfc469-1bdf-4147-9bfd-b0c729988449 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:50.592065 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfc469-1bdf-4147-9bfd-b0c729988449 None admin] RBAC: Adding query filter params (interface=public, service_id=c969f63a9cc9434e9442927e2dbe8c3f, region_id=RegionOne) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:50.592373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfc469-1bdf-4147-9bfd-b0c729988449 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:50.593742 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfc469-1bdf-4147-9bfd-b0c729988449 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:50.601623 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d937c6-2372-41a0-b8ad-c8666c514319 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:50.601912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d937c6-2372-41a0-b8ad-c8666c514319 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:50.602188 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d937c6-2372-41a0-b8ad-c8666c514319 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:50.602665 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 293/583] 10.222.0.13 () {56 vars in 1251 bytes} [Fri Oct 21 21:08:50 2022] GET /identity/v3/endpoints?service_id=c969f63a9cc9434e9442927e2dbe8c3f&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:50.603537 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 291/584] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:08:50.608151 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfc469-1bdf-4147-9bfd-b0c729988449 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:50.609211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77d593-ba7b-4774-bd5e-833a012d4cbf None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:50.609501 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77d593-ba7b-4774-bd5e-833a012d4cbf None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:50.609778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77d593-ba7b-4774-bd5e-833a012d4cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:50.629808 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-5b77d593-ba7b-4774-bd5e-833a012d4cbf None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:50.631020 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-5b77d593-ba7b-4774-bd5e-833a012d4cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:50.719809 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfc469-1bdf-4147-9bfd-b0c729988449 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:50.727036 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 292/585] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 21:08:50.775056 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8dfa1-9d58-48a1-b650-07559f1be76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:50.775885 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8dfa1-9d58-48a1-b650-07559f1be76d None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:50.776162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8dfa1-9d58-48a1-b650-07559f1be76d None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:50.776427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8dfa1-9d58-48a1-b650-07559f1be76d None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:50.777030 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8dfa1-9d58-48a1-b650-07559f1be76d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:50.778378 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8dfa1-9d58-48a1-b650-07559f1be76d None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:50.784688 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 294/586] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Oct 21 21:08:50 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:50.921933 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77d593-ba7b-4774-bd5e-833a012d4cbf None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:51.011132 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:51.027139 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77d593-ba7b-4774-bd5e-833a012d4cbf None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:51.049812 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8dfa1-9d58-48a1-b650-07559f1be76d None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:51.067526 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8bd17-62be-4280-898b-ab17f3c5db98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:51.067913 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:51.068145 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8bd17-62be-4280-898b-ab17f3c5db98 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:51.068428 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8bd17-62be-4280-898b-ab17f3c5db98 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:51.068700 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8bd17-62be-4280-898b-ab17f3c5db98 None admin] PATH_INFO: `/v3/services/network` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:51.069279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8bd17-62be-4280-898b-ab17f3c5db98 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:51.070659 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8bd17-62be-4280-898b-ab17f3c5db98 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:51.079833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-35d8bd17-62be-4280-898b-ab17f3c5db98 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 21:08:51.079833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:51.079833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:08:51.079833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:08:51.079833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:08:51.079833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:08:51.079833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:08:51.079833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:51.079833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 21:08:51.079833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:51.079833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:08:51.079833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:51.079833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:51.079833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:51.079833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:51.079833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:51.079833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:51.080969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:51.080969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:51.080969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:51.080969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:51.080969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:51.080969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:51.080969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:08:51.080969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:08:51.080969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:08:51.080969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:08:51.080969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:51.080969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:51.080969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:51.080969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:51.080969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:51.080969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:51.081929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:51.081929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:51.081929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:51.081929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:51.081929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:51.081929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:51.081929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:51.081929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:51.081929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:51.081929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:51.081929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:08:51.081929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:51.081929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 21:08:51.081929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:51.085945 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 293/587] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:50 2022] GET /identity/v3/services/network => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:51.087218 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8bd17-62be-4280-898b-ab17f3c5db98 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:51.170060 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8dfa1-9d58-48a1-b650-07559f1be76d None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:51.207366 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8447bb-45dc-4294-9e7d-b08138957fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:51.208936 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8447bb-45dc-4294-9e7d-b08138957fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:51.209418 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8447bb-45dc-4294-9e7d-b08138957fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:51.209914 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8447bb-45dc-4294-9e7d-b08138957fd8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:51.221723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8447bb-45dc-4294-9e7d-b08138957fd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:51.227877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-bb8447bb-45dc-4294-9e7d-b08138957fd8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:08:51.227877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:51.227877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:51.227877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:51.227877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:51.227877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:51.227877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:51.227877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:51.227877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:51.227877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:51.227877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:51.227877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:51.227877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:51.227877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:51.227877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:51.227877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:51.228792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:51.228792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:51.228792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:51.228792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:51.228792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:51.228792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:51.228792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:51.228792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:51.228792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:51.228792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:51.228792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:51.228792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:51.228792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:51.228792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:51.228792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:51.228792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:51.229622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:51.229622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:51.229622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:51.229622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:51.229622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:51.229622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:51.229622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:51.229622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:51.229622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:51.229622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:51.229622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:08:51.229622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:51.232364 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 295/588] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:08:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:51.239249 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8447bb-45dc-4294-9e7d-b08138957fd8 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:51.313249 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:51.348876 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8447bb-45dc-4294-9e7d-b08138957fd8 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:51.385751 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947c56d-ac09-4206-a8f6-ad0a4e13a820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:51.387014 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947c56d-ac09-4206-a8f6-ad0a4e13a820 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:51.387510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947c56d-ac09-4206-a8f6-ad0a4e13a820 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:51.388008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947c56d-ac09-4206-a8f6-ad0a4e13a820 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:51.389122 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947c56d-ac09-4206-a8f6-ad0a4e13a820 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:51.389624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947c56d-ac09-4206-a8f6-ad0a4e13a820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:51.391317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947c56d-ac09-4206-a8f6-ad0a4e13a820 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:51.396976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8bd17-62be-4280-898b-ab17f3c5db98 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:51.401041 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 296/589] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:08:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:51.407494 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947c56d-ac09-4206-a8f6-ad0a4e13a820 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:51.504835 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7c96a-4fec-4b8d-80d9-e747ab2b24bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:51.505477 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7c96a-4fec-4b8d-80d9-e747ab2b24bc None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:51.505794 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7c96a-4fec-4b8d-80d9-e747ab2b24bc None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:51.506070 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7c96a-4fec-4b8d-80d9-e747ab2b24bc None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:51.506696 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7c96a-4fec-4b8d-80d9-e747ab2b24bc None admin] RBAC: Adding query filter params (name=network) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:51.506986 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7c96a-4fec-4b8d-80d9-e747ab2b24bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:51.508440 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7c96a-4fec-4b8d-80d9-e747ab2b24bc None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:51.518893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947c56d-ac09-4206-a8f6-ad0a4e13a820 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:51.525896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 294/590] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Oct 21 21:08:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:51.536588 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7c96a-4fec-4b8d-80d9-e747ab2b24bc None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:51.550127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:51.556635 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3c197-0aa9-4ba3-a865-b84fe2b0fc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:51.557871 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3c197-0aa9-4ba3-a865-b84fe2b0fc55 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:51.558382 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3c197-0aa9-4ba3-a865-b84fe2b0fc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:51.558857 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3c197-0aa9-4ba3-a865-b84fe2b0fc55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:51.571344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3c197-0aa9-4ba3-a865-b84fe2b0fc55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:51.573631 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3c197-0aa9-4ba3-a865-b84fe2b0fc55 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:51.573820 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:51.590035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-1fa3c197-0aa9-4ba3-a865-b84fe2b0fc55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:51.590035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:51.590035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:51.590035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:51.590035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:51.590035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:51.590035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:51.590035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:51.590035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:51.590035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:51.590035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:51.590035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:51.590035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:51.590035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:51.590035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:51.590035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:51.591009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:51.591009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:51.591009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:51.591009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:51.591009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:51.591009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:51.591009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:51.591009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:51.591009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:51.591009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:51.591009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:51.591009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:51.591009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:51.591009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:51.591009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:51.591009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:51.591009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:51.591917 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:51.591917 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:51.591917 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:51.592793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 297/591] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:51.599344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3c197-0aa9-4ba3-a865-b84fe2b0fc55 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:51.709492 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3c197-0aa9-4ba3-a865-b84fe2b0fc55 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:51.745273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50425188-2108-45bb-872b-a6024346346a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:51.745963 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50425188-2108-45bb-872b-a6024346346a None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:51.746311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50425188-2108-45bb-872b-a6024346346a None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:51.746649 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50425188-2108-45bb-872b-a6024346346a None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:51.747357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50425188-2108-45bb-872b-a6024346346a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:51.747749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50425188-2108-45bb-872b-a6024346346a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:51.749442 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50425188-2108-45bb-872b-a6024346346a None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:51.762949 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 298/592] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:51.769504 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50425188-2108-45bb-872b-a6024346346a None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:51.828015 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:51.865865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7c96a-4fec-4b8d-80d9-e747ab2b24bc None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:51.937909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50425188-2108-45bb-872b-a6024346346a None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:51.941249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e1712-ed74-4564-bbd2-acfbf8f277f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:51.942128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e1712-ed74-4564-bbd2-acfbf8f277f0 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:51.942621 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e1712-ed74-4564-bbd2-acfbf8f277f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:51.943073 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e1712-ed74-4564-bbd2-acfbf8f277f0 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:51.946931 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e1712-ed74-4564-bbd2-acfbf8f277f0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:51.947423 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e1712-ed74-4564-bbd2-acfbf8f277f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:51.949015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e1712-ed74-4564-bbd2-acfbf8f277f0 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:51.958295 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 295/593] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Oct 21 21:08:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:51.963816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e1712-ed74-4564-bbd2-acfbf8f277f0 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:51.983322 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e8535-be4c-4771-8b8d-c08593348fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:51.984264 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e8535-be4c-4771-8b8d-c08593348fed None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:51.986331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e8535-be4c-4771-8b8d-c08593348fed None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:51.987331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e8535-be4c-4771-8b8d-c08593348fed None admin] PATH_INFO: `/v3/users/swift` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:52.003588 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e8535-be4c-4771-8b8d-c08593348fed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:52.005322 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e8535-be4c-4771-8b8d-c08593348fed None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:52.030037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-3b2e8535-be4c-4771-8b8d-c08593348fed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 21 21:08:52.030037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:52.030037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:52.030037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:52.030037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:52.030037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:52.030037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:52.030037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:52.030037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:52.030037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:52.030037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:52.030037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:52.030037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:08:52.030037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:08:52.030037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:08:52.030037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:52.031037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:52.031985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:52.031985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:52.031985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:08:52.031985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:08:52.031985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:08:52.031985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:08:52.031985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:08:52.031985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:08:52.031985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 21 21:08:52.031985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:52.040472 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 299/594] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:08:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:52.043879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e8535-be4c-4771-8b8d-c08593348fed None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:52.099197 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:52.117422 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e1712-ed74-4564-bbd2-acfbf8f277f0 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:52.154533 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472848f-5cc0-404d-98d8-4b9f8b698ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:52.155833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472848f-5cc0-404d-98d8-4b9f8b698ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:52.159066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472848f-5cc0-404d-98d8-4b9f8b698ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:52.159591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472848f-5cc0-404d-98d8-4b9f8b698ec1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:52.160486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472848f-5cc0-404d-98d8-4b9f8b698ec1 None admin] RBAC: Adding query filter params (interface=public, service_id=e3b4b94f5f8c4101bba8e4287cb9c626, region_id=RegionOne) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:52.162766 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472848f-5cc0-404d-98d8-4b9f8b698ec1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:52.164388 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472848f-5cc0-404d-98d8-4b9f8b698ec1 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:52.177568 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 296/595] 10.222.0.13 () {56 vars in 1251 bytes} [Fri Oct 21 21:08:51 2022] GET /identity/v3/endpoints?service_id=e3b4b94f5f8c4101bba8e4287cb9c626&interface=public®ion_id=RegionOne => generated 194 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:52.183209 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472848f-5cc0-404d-98d8-4b9f8b698ec1 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:52.249119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e8535-be4c-4771-8b8d-c08593348fed None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:52.251387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:52.308683 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbaac12-2280-41ab-ade5-f72f0e168636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:52.309281 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbaac12-2280-41ab-ade5-f72f0e168636 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:52.309559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbaac12-2280-41ab-ade5-f72f0e168636 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:52.309828 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbaac12-2280-41ab-ade5-f72f0e168636 None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:52.310560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbaac12-2280-41ab-ade5-f72f0e168636 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:52.310862 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbaac12-2280-41ab-ade5-f72f0e168636 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:52.312303 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbaac12-2280-41ab-ade5-f72f0e168636 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:52.347817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 300/596] 10.222.0.13 () {56 vars in 1135 bytes} [Fri Oct 21 21:08:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:52.352465 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbaac12-2280-41ab-ade5-f72f0e168636 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:52.387781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472848f-5cc0-404d-98d8-4b9f8b698ec1 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:52.445791 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eecc8-7afc-4b9e-966c-13ee7fe721b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:52.446451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eecc8-7afc-4b9e-966c-13ee7fe721b1 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:52.446661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eecc8-7afc-4b9e-966c-13ee7fe721b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:52.446928 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eecc8-7afc-4b9e-966c-13ee7fe721b1 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:52.447817 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973eecc8-7afc-4b9e-966c-13ee7fe721b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:52.450064 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973eecc8-7afc-4b9e-966c-13ee7fe721b1 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:52.462193 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 297/597] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Oct 21 21:08:52 2022] GET /identity/v3/services => generated 772 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:52.516516 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7282833f-8f51-4ba8-84d3-5f307e84b1c7 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:52.516852 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7282833f-8f51-4ba8-84d3-5f307e84b1c7 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:52.517075 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7282833f-8f51-4ba8-84d3-5f307e84b1c7 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:52.519015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 298/598] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:08:52.525299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74076a-1025-4ed1-9412-b2adb2725dc5 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:52.525894 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74076a-1025-4ed1-9412-b2adb2725dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:52.526362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74076a-1025-4ed1-9412-b2adb2725dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:52.534190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbaac12-2280-41ab-ade5-f72f0e168636 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:52.544507 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:52.557161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-5f74076a-1025-4ed1-9412-b2adb2725dc5 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:52.561045 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-5f74076a-1025-4ed1-9412-b2adb2725dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:52.593793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22585c45-85c3-4f3f-bf3c-f6b366f7cbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:52.594886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22585c45-85c3-4f3f-bf3c-f6b366f7cbab None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:52.595376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22585c45-85c3-4f3f-bf3c-f6b366f7cbab None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:52.595925 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22585c45-85c3-4f3f-bf3c-f6b366f7cbab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:52.628253 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:52.678138 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22585c45-85c3-4f3f-bf3c-f6b366f7cbab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:52.680020 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22585c45-85c3-4f3f-bf3c-f6b366f7cbab None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:52.723065 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 299/599] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:08:52.804274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 301/600] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:08:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 21:08:52.809627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74076a-1025-4ed1-9412-b2adb2725dc5 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:52.922541 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74076a-1025-4ed1-9412-b2adb2725dc5 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:52.961287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5ece2-9c1f-40ea-a49d-e894d86dde12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:52.961926 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5ece2-9c1f-40ea-a49d-e894d86dde12 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:52.962205 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5ece2-9c1f-40ea-a49d-e894d86dde12 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:52.962475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5ece2-9c1f-40ea-a49d-e894d86dde12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:52.965208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:52.973653 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5ece2-9c1f-40ea-a49d-e894d86dde12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:52.975122 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5ece2-9c1f-40ea-a49d-e894d86dde12 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:52.992043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-a9f5ece2-9c1f-40ea-a49d-e894d86dde12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:52.992043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:52.992043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:52.992043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:52.992043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:52.992043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:52.992043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:52.992043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:52.992043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:52.992043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:52.992043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:52.992043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:52.992043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:52.992043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:52.992043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:52.992043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:52.993029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:52.993029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:52.993029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:52.993029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:52.993029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:52.993029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:52.993029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:52.993029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:52.993029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:52.993029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:52.993029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:52.993029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:52.993029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:52.993029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:52.993029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:52.993029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:52.993029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:52.994335 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:52.994335 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:52.994335 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:52.994335 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 300/601] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:52.999281 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22585c45-85c3-4f3f-bf3c-f6b366f7cbab None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:53.032482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5ece2-9c1f-40ea-a49d-e894d86dde12 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:53.129596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:53.135710 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5ece2-9c1f-40ea-a49d-e894d86dde12 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:53.170641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640be77-9bcc-4cb6-b69e-ff20f188142e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:53.171515 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640be77-9bcc-4cb6-b69e-ff20f188142e None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:53.171959 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640be77-9bcc-4cb6-b69e-ff20f188142e None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:53.172397 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640be77-9bcc-4cb6-b69e-ff20f188142e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:53.173169 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640be77-9bcc-4cb6-b69e-ff20f188142e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:53.174752 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640be77-9bcc-4cb6-b69e-ff20f188142e None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:53.183731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-c640be77-9bcc-4cb6-b69e-ff20f188142e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 21:08:53.183731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:53.183731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:08:53.183731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:08:53.183731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:08:53.183731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:08:53.183731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:08:53.183731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:53.183731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 21:08:53.183731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:53.183731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:08:53.183731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:53.183731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:53.183731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:53.183731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:53.183731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:53.184898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:53.184898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:53.184898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:53.184898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:53.184898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:53.184898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:53.184898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:53.184898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:08:53.184898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:08:53.184898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:08:53.184898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:08:53.184898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:53.184898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:53.184898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:53.184898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:53.184898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:53.184898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:53.186015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:53.186015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:53.186015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:53.186015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:53.186015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:53.186015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:53.186015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:53.186015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:53.186015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:53.186015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:53.186015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:08:53.186015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:53.186015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 21:08:53.186015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:53.188408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 301/602] 10.222.0.13 () {56 vars in 1132 bytes} [Fri Oct 21 21:08:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:08:53.190318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24174641-a2f8-42a3-b772-e9b4b7421852 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:53.190732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24174641-a2f8-42a3-b772-e9b4b7421852 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:53.191155 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24174641-a2f8-42a3-b772-e9b4b7421852 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:53.192550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 302/603] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:53.196502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22585c45-85c3-4f3f-bf3c-f6b366f7cbab None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:53.198222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24174641-a2f8-42a3-b772-e9b4b7421852 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:53.237199 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67813ab2-8bb7-49a4-a20b-f09df8b7e4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:53.237818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67813ab2-8bb7-49a4-a20b-f09df8b7e4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:53.238099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67813ab2-8bb7-49a4-a20b-f09df8b7e4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:53.238369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67813ab2-8bb7-49a4-a20b-f09df8b7e4a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:53.239020 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67813ab2-8bb7-49a4-a20b-f09df8b7e4a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:53.239324 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67813ab2-8bb7-49a4-a20b-f09df8b7e4a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:53.240734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67813ab2-8bb7-49a4-a20b-f09df8b7e4a4 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:53.257378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:53.259513 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 302/604] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:53.261205 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa3b98-3b56-4a77-9eae-fad5d3551abe None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:53.261205 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa3b98-3b56-4a77-9eae-fad5d3551abe None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:53.261384 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa3b98-3b56-4a77-9eae-fad5d3551abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:53.287189 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-7cfa3b98-3b56-4a77-9eae-fad5d3551abe None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:53.288753 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-7cfa3b98-3b56-4a77-9eae-fad5d3551abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:53.327910 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24174641-a2f8-42a3-b772-e9b4b7421852 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:53.363980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9243cb-3bdd-495f-a337-50b4c23d5aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:53.364935 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9243cb-3bdd-495f-a337-50b4c23d5aca None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:53.365358 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9243cb-3bdd-495f-a337-50b4c23d5aca None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:53.365773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9243cb-3bdd-495f-a337-50b4c23d5aca None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:53.366536 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9243cb-3bdd-495f-a337-50b4c23d5aca None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:53.366995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9243cb-3bdd-495f-a337-50b4c23d5aca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:53.368509 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 303/605] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:08:53.368676 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9243cb-3bdd-495f-a337-50b4c23d5aca None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:53.369811 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa3b98-3b56-4a77-9eae-fad5d3551abe None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:53.381413 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 303/606] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Oct 21 21:08:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:53.382840 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c333db-ab03-4005-8f00-4e5ba5d5ed6c None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:53.383122 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c333db-ab03-4005-8f00-4e5ba5d5ed6c None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:53.383393 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c333db-ab03-4005-8f00-4e5ba5d5ed6c None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:53.384606 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 304/607] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:53.388671 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c333db-ab03-4005-8f00-4e5ba5d5ed6c None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:53.427335 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:53.484798 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa3b98-3b56-4a77-9eae-fad5d3551abe None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:53.514453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c333db-ab03-4005-8f00-4e5ba5d5ed6c None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:53.521583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5f84b-f5a1-43da-af9a-f326bb744adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:53.522344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5f84b-f5a1-43da-af9a-f326bb744adb None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:53.522728 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5f84b-f5a1-43da-af9a-f326bb744adb None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:53.523101 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5f84b-f5a1-43da-af9a-f326bb744adb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:53.530154 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5f84b-f5a1-43da-af9a-f326bb744adb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:53.531727 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5f84b-f5a1-43da-af9a-f326bb744adb None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:53.543310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-53f5f84b-f5a1-43da-af9a-f326bb744adb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:53.543310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:53.543310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:53.543310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:53.543310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:53.543310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:53.543310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:53.543310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:53.543310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:53.543310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:53.543310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:53.543310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:53.543310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:08:53.543310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:08:53.543310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:08:53.544420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:08:53.544420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:53.544420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:53.544420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:53.544420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:53.544420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:53.544420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:53.544420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:53.544420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:53.544420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:53.544420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:53.544420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:53.544420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:53.544420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:53.544420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:53.544420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:53.544420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:53.545493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:08:53.545493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:08:53.545493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:08:53.545493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:08:53.545493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:08:53.545493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:08:53.545493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:53.545493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:53.546842 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 304/608] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:53.548324 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5c2fd-f66c-44e8-9a86-1fafc010a34f None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:53.548521 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5c2fd-f66c-44e8-9a86-1fafc010a34f None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:53.548898 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5c2fd-f66c-44e8-9a86-1fafc010a34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:53.554238 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:53.559646 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc08df-b8f6-449f-87ba-93fe014aef33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:53.560433 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc08df-b8f6-449f-87ba-93fe014aef33 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:53.560876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc08df-b8f6-449f-87ba-93fe014aef33 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:53.561285 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc08df-b8f6-449f-87ba-93fe014aef33 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:53.562059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc08df-b8f6-449f-87ba-93fe014aef33 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:53.562495 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc08df-b8f6-449f-87ba-93fe014aef33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:53.564592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc08df-b8f6-449f-87ba-93fe014aef33 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:53.569612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-a9d5c2fd-f66c-44e8-9a86-1fafc010a34f None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:53.570150 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 305/609] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Oct 21 21:08:53 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:53.571096 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc08df-b8f6-449f-87ba-93fe014aef33 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:53.576631 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a9d5c2fd-f66c-44e8-9a86-1fafc010a34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:53.718863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 305/610] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:08:53.718863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5c2fd-f66c-44e8-9a86-1fafc010a34f None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:53.761566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc08df-b8f6-449f-87ba-93fe014aef33 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:53.764702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:53.808089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af0e6a-d390-483c-b3b9-f3fd6914cc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:53.809150 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af0e6a-d390-483c-b3b9-f3fd6914cc96 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:53.809566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af0e6a-d390-483c-b3b9-f3fd6914cc96 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:53.809841 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af0e6a-d390-483c-b3b9-f3fd6914cc96 None admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:53.810530 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af0e6a-d390-483c-b3b9-f3fd6914cc96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:53.811000 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af0e6a-d390-483c-b3b9-f3fd6914cc96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:53.813120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af0e6a-d390-483c-b3b9-f3fd6914cc96 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:53.829874 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 306/611] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:08:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:53.830920 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5c2fd-f66c-44e8-9a86-1fafc010a34f None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:53.838038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af0e6a-d390-483c-b3b9-f3fd6914cc96 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:53.872412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51087cde-c2c5-49fc-b141-38360532865a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:53.873166 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51087cde-c2c5-49fc-b141-38360532865a None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:53.873510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51087cde-c2c5-49fc-b141-38360532865a None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:53.873786 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51087cde-c2c5-49fc-b141-38360532865a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:53.885254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51087cde-c2c5-49fc-b141-38360532865a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:53.889936 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:53.890792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-51087cde-c2c5-49fc-b141-38360532865a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:53.890792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:53.890792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:53.890792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:53.890792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:53.890792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:53.890792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:53.890792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:53.890792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:53.890792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:53.890792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:53.890792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:53.890792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:53.890792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:53.890792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:53.890792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:53.891995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:53.891995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:53.891995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:53.891995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:53.891995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:53.891995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:53.891995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:53.891995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:53.891995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:53.891995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:53.891995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:53.891995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:53.891995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:53.891995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:53.891995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:53.891995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:53.892906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:53.892906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:53.892906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:53.892906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:53.892906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:53.892906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:53.892906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:53.892906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:53.892906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:53.892906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:53.892906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:08:53.892906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:53.892906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 306/612] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Oct 21 21:08:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:53.898075 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51087cde-c2c5-49fc-b141-38360532865a None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:53.961470 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af0e6a-d390-483c-b3b9-f3fd6914cc96 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:54.000308 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49649ee1-04fd-4f77-a432-6b4cf66b017e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:54.001733 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49649ee1-04fd-4f77-a432-6b4cf66b017e None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:54.002725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49649ee1-04fd-4f77-a432-6b4cf66b017e None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:54.003229 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49649ee1-04fd-4f77-a432-6b4cf66b017e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:54.004440 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49649ee1-04fd-4f77-a432-6b4cf66b017e None admin] RBAC: Adding query filter params (role.id=c514b065b7604094b899c586f79b0c5b, scope.project.id=8b2c33250ef84d79a4a4742fb4252156, user.id=7ce720c8df014f9a8af062138aecd1b5) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:54.004962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49649ee1-04fd-4f77-a432-6b4cf66b017e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:54.007669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49649ee1-04fd-4f77-a432-6b4cf66b017e None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:54.016217 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 307/613] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:08:53 2022] GET /identity/v3/role_assignments?user.id=7ce720c8df014f9a8af062138aecd1b5&scope.project.id=8b2c33250ef84d79a4a4742fb4252156&role.id=c514b065b7604094b899c586f79b0c5b => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:54.041001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:54.044573 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51087cde-c2c5-49fc-b141-38360532865a None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:54.092151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49649ee1-04fd-4f77-a432-6b4cf66b017e None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:54.100366 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb078f5-800c-43bd-889a-3d50088e300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:54.101005 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb078f5-800c-43bd-889a-3d50088e300a None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:54.101270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb078f5-800c-43bd-889a-3d50088e300a None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:54.107612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb078f5-800c-43bd-889a-3d50088e300a None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:54.107612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb078f5-800c-43bd-889a-3d50088e300a None admin] RBAC: Adding query filter params (name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:54.107612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb078f5-800c-43bd-889a-3d50088e300a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:54.107612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb078f5-800c-43bd-889a-3d50088e300a None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:54.119596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 307/614] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Oct 21 21:08:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:54.119910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb078f5-800c-43bd-889a-3d50088e300a None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:54.255547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49649ee1-04fd-4f77-a432-6b4cf66b017e None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:54.263795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:54.295959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb078f5-800c-43bd-889a-3d50088e300a None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:54.302508 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f0ef6-d7ee-4f13-9b75-f64f7e53e612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:54.303239 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f0ef6-d7ee-4f13-9b75-f64f7e53e612 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:54.303612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f0ef6-d7ee-4f13-9b75-f64f7e53e612 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:54.303945 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f0ef6-d7ee-4f13-9b75-f64f7e53e612 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:54.304592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f0ef6-d7ee-4f13-9b75-f64f7e53e612 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:54.306478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f0ef6-d7ee-4f13-9b75-f64f7e53e612 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:54.317883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-fe2f0ef6-d7ee-4f13-9b75-f64f7e53e612 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 21:08:54.317883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:54.317883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:08:54.317883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:08:54.317883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:08:54.317883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:08:54.317883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:08:54.317883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:54.317883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 21:08:54.317883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:54.317883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:08:54.317883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:54.317883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:54.317883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:54.317883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:54.317883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:54.317883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:54.319138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:54.319138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:54.319138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:54.319138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:54.319138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:54.319138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:54.319138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:08:54.319138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:08:54.319138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:08:54.319138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:08:54.319138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:54.319138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:54.319138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:54.319138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:54.319138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:54.319138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:54.320247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:54.320247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:54.320247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:54.320247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:54.320247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:54.320247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:54.320247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:54.320247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:54.320247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:54.320247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:54.320247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:08:54.320247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:54.320247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 21:08:54.320247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:54.321121 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 308/615] 10.222.0.13 () {56 vars in 1129 bytes} [Fri Oct 21 21:08:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:54.327090 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f0ef6-d7ee-4f13-9b75-f64f7e53e612 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:54.337166 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d182f61-b42c-44de-852b-461fc23ad415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:54.337803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d182f61-b42c-44de-852b-461fc23ad415 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:54.338232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d182f61-b42c-44de-852b-461fc23ad415 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:54.338511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d182f61-b42c-44de-852b-461fc23ad415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:54.351170 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d182f61-b42c-44de-852b-461fc23ad415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:54.352733 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d182f61-b42c-44de-852b-461fc23ad415 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:54.368510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-2d182f61-b42c-44de-852b-461fc23ad415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:54.368510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:54.368510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:54.368510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:54.368510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:54.368510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:54.368510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:54.368510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:54.368510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:54.368510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:54.368510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:54.368510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:54.368510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:54.368510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:54.368510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:54.368510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:54.370718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:54.370718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:54.370718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:54.370718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:54.370718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:54.370718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:54.370718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:54.370718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:54.370718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:54.370718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:54.370718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:54.370718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:54.370718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:54.370718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:54.370718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:54.370718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:54.370718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:54.372899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:54.372899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:54.372899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:54.372899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 308/616] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:54.379596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d182f61-b42c-44de-852b-461fc23ad415 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:54.393654 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:54.438856 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f0ef6-d7ee-4f13-9b75-f64f7e53e612 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:54.486923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d182f61-b42c-44de-852b-461fc23ad415 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:54.522902 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fdbf3-fd20-4530-91d8-30317e539895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:54.523473 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fdbf3-fd20-4530-91d8-30317e539895 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:54.523783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fdbf3-fd20-4530-91d8-30317e539895 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:54.524056 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fdbf3-fd20-4530-91d8-30317e539895 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:54.524754 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fdbf3-fd20-4530-91d8-30317e539895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:54.525156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fdbf3-fd20-4530-91d8-30317e539895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:54.526638 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fdbf3-fd20-4530-91d8-30317e539895 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:54.529073 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53bde2-8596-48ee-865c-46876b56b69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:54.529918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53bde2-8596-48ee-865c-46876b56b69a None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:54.529918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53bde2-8596-48ee-865c-46876b56b69a None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:54.530226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53bde2-8596-48ee-865c-46876b56b69a None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:54.530681 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53bde2-8596-48ee-865c-46876b56b69a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:54.530978 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53bde2-8596-48ee-865c-46876b56b69a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:54.532374 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53bde2-8596-48ee-865c-46876b56b69a None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:54.540895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 309/617] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:54.546298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fdbf3-fd20-4530-91d8-30317e539895 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:54.549013 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 309/618] 10.222.0.13 () {56 vars in 1124 bytes} [Fri Oct 21 21:08:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:54.554350 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53bde2-8596-48ee-865c-46876b56b69a None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:54.627577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:54.652448 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fdbf3-fd20-4530-91d8-30317e539895 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:54.676910 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:54.692739 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e49ca-7693-4ddf-acce-97ff19656076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:54.693373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e49ca-7693-4ddf-acce-97ff19656076 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:54.693647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e49ca-7693-4ddf-acce-97ff19656076 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:54.693976 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e49ca-7693-4ddf-acce-97ff19656076 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:54.702224 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e49ca-7693-4ddf-acce-97ff19656076 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:54.703689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e49ca-7693-4ddf-acce-97ff19656076 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:54.717127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-657e49ca-7693-4ddf-acce-97ff19656076 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 21 21:08:54.717127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:54.717127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:54.717127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:54.717127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:54.717127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:54.717127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:54.717127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:54.717127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:54.717127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:54.717127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:54.717127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:54.717127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:08:54.717127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:08:54.717127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:08:54.717127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:54.719496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:54.734469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:54.734469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:54.734469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:08:54.734469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:08:54.734469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:08:54.734469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:08:54.734469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:08:54.734469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:08:54.734469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 21 21:08:54.734469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:54.734469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 310/619] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Oct 21 21:08:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:08:54.734469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e49ca-7693-4ddf-acce-97ff19656076 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:54.840546 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e49ca-7693-4ddf-acce-97ff19656076 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:54.878212 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a9747-f62a-4f29-8ed3-edf22d4d2335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:54.878814 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a9747-f62a-4f29-8ed3-edf22d4d2335 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:54.879094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a9747-f62a-4f29-8ed3-edf22d4d2335 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:54.879359 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a9747-f62a-4f29-8ed3-edf22d4d2335 None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:54.880194 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a9747-f62a-4f29-8ed3-edf22d4d2335 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:54.880501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a9747-f62a-4f29-8ed3-edf22d4d2335 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:54.882072 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a9747-f62a-4f29-8ed3-edf22d4d2335 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:54.901337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 311/620] 10.222.0.13 () {56 vars in 1149 bytes} [Fri Oct 21 21:08:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:54.901720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:54.907520 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a9747-f62a-4f29-8ed3-edf22d4d2335 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:54.961487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53bde2-8596-48ee-865c-46876b56b69a None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:55.025633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a9747-f62a-4f29-8ed3-edf22d4d2335 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:55.072679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715c122-4a82-4eb4-96fc-f354e80d9470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:55.073679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715c122-4a82-4eb4-96fc-f354e80d9470 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:55.073962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715c122-4a82-4eb4-96fc-f354e80d9470 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:55.074232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715c122-4a82-4eb4-96fc-f354e80d9470 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:55.079514 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed093c4-7c7a-4512-9296-fa21cd896275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:55.080828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed093c4-7c7a-4512-9296-fa21cd896275 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:55.081106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed093c4-7c7a-4512-9296-fa21cd896275 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:55.081374 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed093c4-7c7a-4512-9296-fa21cd896275 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:55.082004 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed093c4-7c7a-4512-9296-fa21cd896275 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:55.082303 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed093c4-7c7a-4512-9296-fa21cd896275 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:55.083709 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed093c4-7c7a-4512-9296-fa21cd896275 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:55.099657 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 310/621] 10.222.0.13 () {56 vars in 1124 bytes} [Fri Oct 21 21:08:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 545 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:55.117491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed093c4-7c7a-4512-9296-fa21cd896275 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:55.171029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715c122-4a82-4eb4-96fc-f354e80d9470 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:55.173036 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715c122-4a82-4eb4-96fc-f354e80d9470 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:55.196292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:55.233838 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:55.241151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed093c4-7c7a-4512-9296-fa21cd896275 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:55.275532 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd2930-482c-442c-a5f5-c754e285a1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:55.276168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd2930-482c-442c-a5f5-c754e285a1ad None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:55.276499 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd2930-482c-442c-a5f5-c754e285a1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:55.276778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd2930-482c-442c-a5f5-c754e285a1ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:55.277436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd2930-482c-442c-a5f5-c754e285a1ad None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:55.279118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd2930-482c-442c-a5f5-c754e285a1ad None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:55.304545 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 312/622] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:08:54 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 21:08:55.306444 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5b26b-0b59-4947-b32b-5d95d7b4e022 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:55.306956 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5b26b-0b59-4947-b32b-5d95d7b4e022 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:55.307430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5b26b-0b59-4947-b32b-5d95d7b4e022 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:55.313007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 311/624] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:08:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:08:55.313217 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd2930-482c-442c-a5f5-c754e285a1ad None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:55.313411 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 313/624] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:55.315279 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7c36a-50a0-46de-b229-cd253905de1f None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:55.315839 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7c36a-50a0-46de-b229-cd253905de1f None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:55.316448 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7c36a-50a0-46de-b229-cd253905de1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:55.336577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-7ee7c36a-50a0-46de-b229-cd253905de1f None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:55.338187 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-7ee7c36a-50a0-46de-b229-cd253905de1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:55.458649 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 314/625] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:08:55.478360 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd2930-482c-442c-a5f5-c754e285a1ad None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:55.537610 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa2925-e54f-464c-b81b-8e2d3815e68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:55.538243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa2925-e54f-464c-b81b-8e2d3815e68a None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:55.538526 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa2925-e54f-464c-b81b-8e2d3815e68a None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:55.538797 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa2925-e54f-464c-b81b-8e2d3815e68a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:55.550666 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa2925-e54f-464c-b81b-8e2d3815e68a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:55.552182 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa2925-e54f-464c-b81b-8e2d3815e68a None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:55.568249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-cbaa2925-e54f-464c-b81b-8e2d3815e68a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:55.568249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:55.568249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:55.568249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:55.568249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:55.568249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:55.568249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:55.568249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:55.568249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:55.568249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:55.568249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:55.568249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:55.568249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:08:55.568249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:08:55.568249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:08:55.568249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:08:55.569247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:55.569247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:55.569247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:55.569247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:55.569247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:55.569247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:55.569247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:55.569247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:55.569247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:55.569247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:55.569247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:55.569247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:55.569247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:55.569247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:55.569247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:55.569247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:55.569247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:08:55.570130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:08:55.570130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:08:55.570130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:55.570130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 312/626] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:55.576090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7c36a-50a0-46de-b229-cd253905de1f None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:55.598467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:55.702459 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7c36a-50a0-46de-b229-cd253905de1f None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:55.769836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971c082-0918-4f9c-a337-65bb4df704cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:55.770449 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971c082-0918-4f9c-a337-65bb4df704cc None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:55.770730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971c082-0918-4f9c-a337-65bb4df704cc None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:55.771004 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971c082-0918-4f9c-a337-65bb4df704cc None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:55.771671 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971c082-0918-4f9c-a337-65bb4df704cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:55.771979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971c082-0918-4f9c-a337-65bb4df704cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:55.773336 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971c082-0918-4f9c-a337-65bb4df704cc None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:55.789966 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 315/627] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:55.795611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa2925-e54f-464c-b81b-8e2d3815e68a None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:55.800359 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:55.814029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971c082-0918-4f9c-a337-65bb4df704cc None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:55.927879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:55.956562 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971c082-0918-4f9c-a337-65bb4df704cc None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:56.000187 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa2925-e54f-464c-b81b-8e2d3815e68a None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:56.011681 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190405ce-57b5-49a5-977c-e95133378938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:56.012631 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190405ce-57b5-49a5-977c-e95133378938 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:56.013411 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190405ce-57b5-49a5-977c-e95133378938 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:56.013880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190405ce-57b5-49a5-977c-e95133378938 None admin] PATH_INFO: `/v3/services/network` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:56.014668 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190405ce-57b5-49a5-977c-e95133378938 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:56.016253 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190405ce-57b5-49a5-977c-e95133378938 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:56.030991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-190405ce-57b5-49a5-977c-e95133378938 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 21:08:56.030991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:56.030991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:08:56.030991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:08:56.030991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:08:56.030991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:08:56.030991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:08:56.030991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:56.030991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 21:08:56.030991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:56.030991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:08:56.030991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:56.030991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:56.030991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:56.030991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:56.030991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:56.030991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:56.032067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:56.032067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:56.032067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:56.032067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:56.032067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:56.032067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:56.032067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:08:56.032067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:08:56.032067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:08:56.032067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:08:56.032067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:56.032067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:56.032067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:56.032067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:56.032067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:56.032067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:56.032909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:56.032909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:56.032909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:56.032909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:56.032909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:56.032909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:56.032909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:56.032909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:56.032909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:56.032909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:56.032909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:08:56.032909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:56.032909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 21:08:56.032909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:56.036183 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 316/628] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:55 2022] GET /identity/v3/services/network => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:08:56.043616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190405ce-57b5-49a5-977c-e95133378938 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:56.084224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f55a2-210a-4e7d-be15-c2ac9aaa1752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:56.084855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f55a2-210a-4e7d-be15-c2ac9aaa1752 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:56.085130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f55a2-210a-4e7d-be15-c2ac9aaa1752 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:56.085399 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f55a2-210a-4e7d-be15-c2ac9aaa1752 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:56.098833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f55a2-210a-4e7d-be15-c2ac9aaa1752 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:56.108386 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f55a2-210a-4e7d-be15-c2ac9aaa1752 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:56.132465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-248f55a2-210a-4e7d-be15-c2ac9aaa1752 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:56.132465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:56.132465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:56.132465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:56.132465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:56.132465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:56.132465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:56.132465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:56.132465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:56.132465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:56.132465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:56.132465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:56.132465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:08:56.132465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:08:56.132465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:08:56.133615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:08:56.133615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:56.133615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:56.133615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:56.133615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:56.133615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:56.133615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:56.133615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:56.133615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:56.133615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:56.133615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:56.133615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:56.133615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:56.133615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:56.133615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:56.133615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:56.133615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:56.134812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:08:56.134812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:08:56.134812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:08:56.134812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:08:56.134812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:08:56.134812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:08:56.134812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:08:56.134812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:56.134812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 313/629] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:08:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:56.142033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f55a2-210a-4e7d-be15-c2ac9aaa1752 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:56.156764 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190405ce-57b5-49a5-977c-e95133378938 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:56.193299 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580c86-b6f7-4ada-bdef-db8c3be0e11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:56.194189 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580c86-b6f7-4ada-bdef-db8c3be0e11b None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:56.194617 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580c86-b6f7-4ada-bdef-db8c3be0e11b None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:56.195035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580c86-b6f7-4ada-bdef-db8c3be0e11b None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:56.195943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51580c86-b6f7-4ada-bdef-db8c3be0e11b None admin] RBAC: Adding query filter params (name=network) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:56.196461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51580c86-b6f7-4ada-bdef-db8c3be0e11b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:56.198032 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51580c86-b6f7-4ada-bdef-db8c3be0e11b None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:56.203375 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 317/630] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Oct 21 21:08:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:56.208531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580c86-b6f7-4ada-bdef-db8c3be0e11b None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:56.220066 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:56.276074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:56.317904 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f55a2-210a-4e7d-be15-c2ac9aaa1752 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:56.339492 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580c86-b6f7-4ada-bdef-db8c3be0e11b None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:56.355778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6170b5c-2be3-4386-a3b9-4757e7589bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:56.356394 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6170b5c-2be3-4386-a3b9-4757e7589bda None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:56.356923 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6170b5c-2be3-4386-a3b9-4757e7589bda None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:56.357198 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6170b5c-2be3-4386-a3b9-4757e7589bda None admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:56.358282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6170b5c-2be3-4386-a3b9-4757e7589bda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:56.358592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6170b5c-2be3-4386-a3b9-4757e7589bda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:56.360066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6170b5c-2be3-4386-a3b9-4757e7589bda None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:56.375601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded29d6-37ed-4d59-8744-b7543ed36d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:56.376256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded29d6-37ed-4d59-8744-b7543ed36d62 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:56.376533 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded29d6-37ed-4d59-8744-b7543ed36d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:56.376807 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded29d6-37ed-4d59-8744-b7543ed36d62 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:56.377570 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded29d6-37ed-4d59-8744-b7543ed36d62 None admin] RBAC: Adding query filter params (type=network) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:56.377879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded29d6-37ed-4d59-8744-b7543ed36d62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:56.379299 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded29d6-37ed-4d59-8744-b7543ed36d62 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:56.381730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 314/631] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:08:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:56.383857 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c2b99-7b18-412d-939e-0250064c4179 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:56.384151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c2b99-7b18-412d-939e-0250064c4179 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:56.384430 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c2b99-7b18-412d-939e-0250064c4179 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:56.385694 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 318/632] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Oct 21 21:08:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:08:56.385820 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 315/633] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:56.386559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded29d6-37ed-4d59-8744-b7543ed36d62 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:56.393318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a6e02-5677-43ca-94d4-1f10bf37c4fc None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:56.393601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a6e02-5677-43ca-94d4-1f10bf37c4fc None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:56.393873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a6e02-5677-43ca-94d4-1f10bf37c4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:56.416802 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-ee2a6e02-5677-43ca-94d4-1f10bf37c4fc None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:56.418680 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ee2a6e02-5677-43ca-94d4-1f10bf37c4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:56.465461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:56.503830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded29d6-37ed-4d59-8744-b7543ed36d62 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:56.523996 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 316/634] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 21:08:56.523996 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a6e02-5677-43ca-94d4-1f10bf37c4fc None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:56.564404 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc694ca-5483-471f-8507-555781ce2878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:56.565039 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc694ca-5483-471f-8507-555781ce2878 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:56.565321 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc694ca-5483-471f-8507-555781ce2878 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:56.565590 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc694ca-5483-471f-8507-555781ce2878 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:56.566318 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc694ca-5483-471f-8507-555781ce2878 None admin] RBAC: Adding query filter params (role.id=b0f9a090148d4a0eb6456472edec3d5a, scope.project.id=8b2c33250ef84d79a4a4742fb4252156, user.id=2346003afe74477496045e9bc72173d5) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:56.566622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc694ca-5483-471f-8507-555781ce2878 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:56.568069 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc694ca-5483-471f-8507-555781ce2878 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:56.574631 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 319/635] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:08:56 2022] GET /identity/v3/role_assignments?user.id=2346003afe74477496045e9bc72173d5&scope.project.id=8b2c33250ef84d79a4a4742fb4252156&role.id=b0f9a090148d4a0eb6456472edec3d5a => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:56.590007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5102d9-a05e-4ad0-b27e-a0851d0729c3 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:56.590357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5102d9-a05e-4ad0-b27e-a0851d0729c3 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:56.590554 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5102d9-a05e-4ad0-b27e-a0851d0729c3 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:56.592850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 320/636] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:56.600703 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7655e-376b-43fe-a2b4-a9dc82b71a33 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:56.601105 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7655e-376b-43fe-a2b4-a9dc82b71a33 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:56.601485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7655e-376b-43fe-a2b4-a9dc82b71a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:56.617578 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:56.627658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-e9c7655e-376b-43fe-a2b4-a9dc82b71a33 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:56.629215 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-e9c7655e-376b-43fe-a2b4-a9dc82b71a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:56.670205 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a6e02-5677-43ca-94d4-1f10bf37c4fc None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:56.713949 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7c55f-b494-4c30-bf1d-a0694bd0401c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:56.715147 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7c55f-b494-4c30-bf1d-a0694bd0401c None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:56.715795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7c55f-b494-4c30-bf1d-a0694bd0401c None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:56.716257 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7c55f-b494-4c30-bf1d-a0694bd0401c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:56.717317 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7c55f-b494-4c30-bf1d-a0694bd0401c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:56.719415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7c55f-b494-4c30-bf1d-a0694bd0401c None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:56.737410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 321/637] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 21:08:56.765950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 317/638] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:08:56 2022] POST /identity/v3/endpoints => generated 337 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:08:56.894714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7655e-376b-43fe-a2b4-a9dc82b71a33 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:56.969051 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:57.011967 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7655e-376b-43fe-a2b4-a9dc82b71a33 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:57.053854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1aeea-286b-4c5d-ba59-3e4f7ab8a7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:57.054468 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1aeea-286b-4c5d-ba59-3e4f7ab8a7fd None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:57.054746 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1aeea-286b-4c5d-ba59-3e4f7ab8a7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:57.055021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1aeea-286b-4c5d-ba59-3e4f7ab8a7fd None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:57.055785 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1aeea-286b-4c5d-ba59-3e4f7ab8a7fd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:57.056509 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7c55f-b494-4c30-bf1d-a0694bd0401c None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:57.057250 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1aeea-286b-4c5d-ba59-3e4f7ab8a7fd None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:57.077957 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 322/639] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Oct 21 21:08:56 2022] POST /identity/v3/services => generated 269 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:08:57.195529 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7c55f-b494-4c30-bf1d-a0694bd0401c None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:57.208668 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:57.245309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80bd87-41b0-4ece-a5a3-be078d861aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:57.245964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80bd87-41b0-4ece-a5a3-be078d861aae None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:57.246248 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80bd87-41b0-4ece-a5a3-be078d861aae None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:57.246520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80bd87-41b0-4ece-a5a3-be078d861aae None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:57.247104 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80bd87-41b0-4ece-a5a3-be078d861aae None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:57.248556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80bd87-41b0-4ece-a5a3-be078d861aae None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:57.260256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-4e80bd87-41b0-4ece-a5a3-be078d861aae None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 21:08:57.260256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:57.260256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:08:57.260256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:08:57.260256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:08:57.260256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:08:57.260256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:08:57.260256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:57.260256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 21:08:57.260256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:57.260256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:08:57.260256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:57.260256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:57.260256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:57.260256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:57.260256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:57.260256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:57.261410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:57.261410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:57.261410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:57.261410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:57.261410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:57.261410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:57.261410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:08:57.261410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:08:57.261410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:08:57.261410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:08:57.261410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:57.261410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:57.261410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:57.261410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:57.261410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:57.261410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:57.262995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:57.262995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:57.262995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:57.262995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:57.262995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:57.262995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:57.262995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:57.262995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:57.262995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:57.262995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:57.262995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:08:57.262995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:57.262995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 21:08:57.262995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:57.262995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 318/640] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Oct 21 21:08:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:57.268308 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1aeea-286b-4c5d-ba59-3e4f7ab8a7fd None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:57.386631 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1aeea-286b-4c5d-ba59-3e4f7ab8a7fd None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:57.408791 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:57.424176 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597ef67-0e96-404b-8a6c-b7ce4fa410ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:57.424801 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597ef67-0e96-404b-8a6c-b7ce4fa410ed None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:57.425076 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597ef67-0e96-404b-8a6c-b7ce4fa410ed None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:57.425349 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597ef67-0e96-404b-8a6c-b7ce4fa410ed None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:57.425979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597ef67-0e96-404b-8a6c-b7ce4fa410ed None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:57.426282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597ef67-0e96-404b-8a6c-b7ce4fa410ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:57.427650 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597ef67-0e96-404b-8a6c-b7ce4fa410ed None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:57.433155 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 323/641] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Oct 21 21:08:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:57.438452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80bd87-41b0-4ece-a5a3-be078d861aae None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:57.546815 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80bd87-41b0-4ece-a5a3-be078d861aae None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:57.582736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b223d4-f2cc-413e-9638-b6f0c6b3883d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:57.583418 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b223d4-f2cc-413e-9638-b6f0c6b3883d None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:57.583735 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b223d4-f2cc-413e-9638-b6f0c6b3883d None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:57.584010 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b223d4-f2cc-413e-9638-b6f0c6b3883d None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:57.584639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b223d4-f2cc-413e-9638-b6f0c6b3883d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:57.584945 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b223d4-f2cc-413e-9638-b6f0c6b3883d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:57.586335 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b223d4-f2cc-413e-9638-b6f0c6b3883d None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:57.591354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 319/642] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Oct 21 21:08:57 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:57.946998 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4f7f2-72c0-46a4-ae09-4f244364aae0 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:57.947360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4f7f2-72c0-46a4-ae09-4f244364aae0 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:57.947553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4f7f2-72c0-46a4-ae09-4f244364aae0 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:57.949231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 324/643] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:57.954534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62b41b-36d7-467a-bb2d-7734ca9c01f6 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:57.955551 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62b41b-36d7-467a-bb2d-7734ca9c01f6 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:57.955653 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62b41b-36d7-467a-bb2d-7734ca9c01f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:57.973891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-de62b41b-36d7-467a-bb2d-7734ca9c01f6 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:57.975072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-de62b41b-36d7-467a-bb2d-7734ca9c01f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:58.002873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:58.051684 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 320/644] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:57 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:08:58.315176 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4f7f2-72c0-46a4-ae09-4f244364aae0 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:58.420505 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4f7f2-72c0-46a4-ae09-4f244364aae0 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:58.457378 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fd506-20c0-4f18-b72c-b966bddf00b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:58.458375 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fd506-20c0-4f18-b72c-b966bddf00b1 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:58.458839 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fd506-20c0-4f18-b72c-b966bddf00b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:58.459355 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fd506-20c0-4f18-b72c-b966bddf00b1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:58.460172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550fd506-20c0-4f18-b72c-b966bddf00b1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:58.463315 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550fd506-20c0-4f18-b72c-b966bddf00b1 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:58.474239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-550fd506-20c0-4f18-b72c-b966bddf00b1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 21:08:58.474239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:58.474239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:08:58.474239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:08:58.474239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:08:58.474239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:08:58.474239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:08:58.474239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:58.474239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 21:08:58.474239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:58.474239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:08:58.474239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:58.474239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:58.474239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:58.474239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:58.474239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:58.474239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:58.475291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:58.475291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:58.475291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:58.475291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:58.475291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:58.475291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:58.475291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:08:58.475291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:08:58.475291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:08:58.475291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:08:58.475291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:58.475291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:58.475291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:58.475291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:58.475291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:58.475291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:58.476173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:58.476173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:58.476173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:58.476173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:58.476173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:58.476173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:58.476173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:58.476173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:58.476173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:58.476173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:58.476173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:08:58.476173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:08:58.476173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 21:08:58.476173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:08:58.478585 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 325/645] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Oct 21 21:08:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:58.484348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62b41b-36d7-467a-bb2d-7734ca9c01f6 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:58.588847 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62b41b-36d7-467a-bb2d-7734ca9c01f6 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:58.623229 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a78bb5-892c-458c-894c-2dfb5f0bdea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:58.623883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a78bb5-892c-458c-894c-2dfb5f0bdea3 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:58.624164 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a78bb5-892c-458c-894c-2dfb5f0bdea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:58.624441 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a78bb5-892c-458c-894c-2dfb5f0bdea3 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:58.625056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a78bb5-892c-458c-894c-2dfb5f0bdea3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:58.625357 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a78bb5-892c-458c-894c-2dfb5f0bdea3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:58.626721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a78bb5-892c-458c-894c-2dfb5f0bdea3 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:58.635679 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 321/646] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Oct 21 21:08:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:58.637456 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fd506-20c0-4f18-b72c-b966bddf00b1 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:58.692914 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:58.745818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fd506-20c0-4f18-b72c-b966bddf00b1 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:58.780063 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae385e-9d10-4e65-b59d-1e6183e1e70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:58.780679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae385e-9d10-4e65-b59d-1e6183e1e70d None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:58.780960 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae385e-9d10-4e65-b59d-1e6183e1e70d None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:58.781232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae385e-9d10-4e65-b59d-1e6183e1e70d None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:58.781857 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae385e-9d10-4e65-b59d-1e6183e1e70d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:58.782159 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae385e-9d10-4e65-b59d-1e6183e1e70d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:58.783598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae385e-9d10-4e65-b59d-1e6183e1e70d None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:58.790236 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 326/647] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Oct 21 21:08:58 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:59.092516 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf4cf9-93fb-4c6e-8a2d-82e1169c9198 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:59.093036 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf4cf9-93fb-4c6e-8a2d-82e1169c9198 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:59.093036 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf4cf9-93fb-4c6e-8a2d-82e1169c9198 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:59.094763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 322/648] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:08:59.101044 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cb6f0-35a5-43f3-8623-3c9e6107ca73 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:59.101330 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cb6f0-35a5-43f3-8623-3c9e6107ca73 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:59.101602 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cb6f0-35a5-43f3-8623-3c9e6107ca73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:59.121086 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-733cb6f0-35a5-43f3-8623-3c9e6107ca73 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:59.122631 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-733cb6f0-35a5-43f3-8623-3c9e6107ca73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:59.193813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 327/649] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:59 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:08:59.481958 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf4cf9-93fb-4c6e-8a2d-82e1169c9198 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:59.503697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461087b7-a4d5-4dfb-89c7-223ed2891185 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:59.504667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461087b7-a4d5-4dfb-89c7-223ed2891185 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:59.505143 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461087b7-a4d5-4dfb-89c7-223ed2891185 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:59.507067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 328/650] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:08:59.512833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:59.513616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287d927-0884-4fc0-961a-a3d4b8ec35b7 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:59.514733 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287d927-0884-4fc0-961a-a3d4b8ec35b7 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:59.515313 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287d927-0884-4fc0-961a-a3d4b8ec35b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:59.536428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-5287d927-0884-4fc0-961a-a3d4b8ec35b7 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:59.537932 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-5287d927-0884-4fc0-961a-a3d4b8ec35b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:59.586926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:59.599841 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf4cf9-93fb-4c6e-8a2d-82e1169c9198 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:59.635635 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 329/651] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:59 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 21:08:59.650064 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2a943-1f42-4631-8890-f59ff82c7234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:59.651162 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2a943-1f42-4631-8890-f59ff82c7234 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:59.651685 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2a943-1f42-4631-8890-f59ff82c7234 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:59.652176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2a943-1f42-4631-8890-f59ff82c7234 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:59.665841 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2a943-1f42-4631-8890-f59ff82c7234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:08:59.671226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-23f2a943-1f42-4631-8890-f59ff82c7234 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 21 21:08:59.671226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:08:59.671226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:08:59.671226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:08:59.671226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:08:59.671226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:08:59.671226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:08:59.671226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:08:59.671226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:08:59.671226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:08:59.671226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:08:59.671226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:08:59.671226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:08:59.671226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:08:59.671226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:08:59.671226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:08:59.672309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:08:59.672309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:08:59.672309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:08:59.672309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:08:59.672309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:08:59.672309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:08:59.672309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:08:59.672309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:08:59.672309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:08:59.672309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:08:59.672309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:08:59.672309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:08:59.672309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:08:59.672309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:08:59.672309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:08:59.672309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:08:59.673136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:08:59.673136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:08:59.673136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:08:59.673136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:08:59.673136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:08:59.673136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:08:59.673136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:08:59.673136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:08:59.673136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:08:59.673136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:08:59.673136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 21 21:08:59.673136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:08:59.676590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 323/652] 10.222.0.13 () {56 vars in 1120 bytes} [Fri Oct 21 21:08:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:08:59.682686 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287d927-0884-4fc0-961a-a3d4b8ec35b7 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:59.764470 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2d3f9-f4fd-48db-b183-3a0e28610ae1 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:59.764797 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2d3f9-f4fd-48db-b183-3a0e28610ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:59.765031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2d3f9-f4fd-48db-b183-3a0e28610ae1 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:59.767185 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 324/653] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:08:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:08:59.772208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c6e0c-dbe6-4330-b560-4e12665250e7 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:59.772494 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c6e0c-dbe6-4330-b560-4e12665250e7 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:59.772764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c6e0c-dbe6-4330-b560-4e12665250e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:59.793418 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-4e7c6e0c-dbe6-4330-b560-4e12665250e7 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:08:59.794591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-4e7c6e0c-dbe6-4330-b560-4e12665250e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:08:59.795061 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287d927-0884-4fc0-961a-a3d4b8ec35b7 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:08:59.841925 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb892bd-1d5c-42f4-b162-d21a2a958030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:08:59.843307 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb892bd-1d5c-42f4-b162-d21a2a958030 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:08:59.843851 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb892bd-1d5c-42f4-b162-d21a2a958030 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:08:59.844333 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb892bd-1d5c-42f4-b162-d21a2a958030 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:08:59.846078 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb892bd-1d5c-42f4-b162-d21a2a958030 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:08:59.846654 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb892bd-1d5c-42f4-b162-d21a2a958030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:08:59.848802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb892bd-1d5c-42f4-b162-d21a2a958030 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:08:59.859313 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 330/654] 10.222.0.13 () {56 vars in 1115 bytes} [Fri Oct 21 21:08:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:08:59.864544 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb892bd-1d5c-42f4-b162-d21a2a958030 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:59.872952 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 325/655] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:08:59 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:08:59.903164 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c6e0c-dbe6-4330-b560-4e12665250e7 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:08:59.906202 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:59.931826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:08:59.991422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb892bd-1d5c-42f4-b162-d21a2a958030 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:00.031303 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c6e0c-dbe6-4330-b560-4e12665250e7 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:00.035243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb0c86-c9d6-4047-8b41-646607a94d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:00.036014 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb0c86-c9d6-4047-8b41-646607a94d86 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:00.036388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb0c86-c9d6-4047-8b41-646607a94d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:00.036759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb0c86-c9d6-4047-8b41-646607a94d86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:00.049083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb0c86-c9d6-4047-8b41-646607a94d86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:00.050760 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb0c86-c9d6-4047-8b41-646607a94d86 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:00.066082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-97fb0c86-c9d6-4047-8b41-646607a94d86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:00.066082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:00.066082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:00.066082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:00.066082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:00.066082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:00.066082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:00.066082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:00.066082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:00.066082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:00.066082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:00.066082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:00.066082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:09:00.066082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:09:00.066082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:09:00.066082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:09:00.067288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:00.067288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:00.067288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:00.067288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:00.067288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:00.067288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:00.067288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:00.067288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:00.067288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:00.067288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:00.067288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:00.067288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:00.067288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:00.067288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:00.067288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:00.067288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:00.067288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:09:00.069183 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:09:00.069183 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:00.069183 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:00.069565 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4baa50-afd3-474b-a8d2-93239020458c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:00.069751 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 331/656] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:08:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:00.073138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4baa50-afd3-474b-a8d2-93239020458c None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:00.073138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4baa50-afd3-474b-a8d2-93239020458c None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:00.073138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4baa50-afd3-474b-a8d2-93239020458c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:00.073138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4baa50-afd3-474b-a8d2-93239020458c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:00.074301 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4baa50-afd3-474b-a8d2-93239020458c None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:00.075482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb0c86-c9d6-4047-8b41-646607a94d86 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:00.087417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-8a4baa50-afd3-474b-a8d2-93239020458c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 21:09:00.087417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:00.087417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:09:00.087417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:09:00.087417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:09:00.087417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:09:00.087417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:09:00.087417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:00.087417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 21:09:00.087417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:00.087417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:00.087417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:00.087417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:00.087417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:00.087417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:00.087417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:00.088546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:00.088546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:00.088546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:00.088546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:00.088546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:00.088546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:00.088546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:00.088546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:09:00.088546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:09:00.088546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:09:00.088546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:09:00.088546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:00.088546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:00.088546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:00.088546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:00.088546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:00.088546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:00.089436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:00.089436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:00.089436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:00.089436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:00.089436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:00.089436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:00.089436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:00.089436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:00.089436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:00.089436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:00.089436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:09:00.089436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:00.089436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 21:09:00.089436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:00.091644 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 326/657] 10.222.0.13 () {56 vars in 1132 bytes} [Fri Oct 21 21:08:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:09:00.094482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4baa50-afd3-474b-a8d2-93239020458c None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:00.183598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:00.195933 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:00.199393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb0c86-c9d6-4047-8b41-646607a94d86 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:00.215818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4baa50-afd3-474b-a8d2-93239020458c None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:00.234838 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3967bb-edaa-4291-833c-1d91cbf84b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:00.235445 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3967bb-edaa-4291-833c-1d91cbf84b00 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:00.235758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3967bb-edaa-4291-833c-1d91cbf84b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:00.236784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3967bb-edaa-4291-833c-1d91cbf84b00 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:00.237174 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3967bb-edaa-4291-833c-1d91cbf84b00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:00.237479 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3967bb-edaa-4291-833c-1d91cbf84b00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:00.238864 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3967bb-edaa-4291-833c-1d91cbf84b00 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:00.250712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 332/658] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:00.251336 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3967bb-edaa-4291-833c-1d91cbf84b00 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:00.256485 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bfa78-fb38-476c-8e78-2edf52205080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:00.257103 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bfa78-fb38-476c-8e78-2edf52205080 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:00.257379 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bfa78-fb38-476c-8e78-2edf52205080 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:00.257647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bfa78-fb38-476c-8e78-2edf52205080 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:00.258268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772bfa78-fb38-476c-8e78-2edf52205080 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:00.258568 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772bfa78-fb38-476c-8e78-2edf52205080 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:00.259959 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772bfa78-fb38-476c-8e78-2edf52205080 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:00.265030 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 327/659] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Oct 21 21:09:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:09:00.265030 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bfa78-fb38-476c-8e78-2edf52205080 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:00.351354 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3967bb-edaa-4291-833c-1d91cbf84b00 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:00.375098 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bfa78-fb38-476c-8e78-2edf52205080 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:00.385697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d6fbd-f286-46a7-819a-ddd5c2f4cdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:00.386309 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d6fbd-f286-46a7-819a-ddd5c2f4cdf4 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:00.386592 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d6fbd-f286-46a7-819a-ddd5c2f4cdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:00.386861 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d6fbd-f286-46a7-819a-ddd5c2f4cdf4 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:00.387612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d6fbd-f286-46a7-819a-ddd5c2f4cdf4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:00.389145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d6fbd-f286-46a7-819a-ddd5c2f4cdf4 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:00.404650 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 333/660] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Oct 21 21:09:00 2022] POST /identity/v3/services => generated 241 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:09:00.405469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d6fbd-f286-46a7-819a-ddd5c2f4cdf4 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:00.426857 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c6b0f-207c-4b89-9436-b4c2dad1f8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:00.428244 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c6b0f-207c-4b89-9436-b4c2dad1f8ed None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:00.429054 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c6b0f-207c-4b89-9436-b4c2dad1f8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:00.429581 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c6b0f-207c-4b89-9436-b4c2dad1f8ed None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:00.430742 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:00.442282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c6b0f-207c-4b89-9436-b4c2dad1f8ed None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:00.443796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c6b0f-207c-4b89-9436-b4c2dad1f8ed None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:00.456836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-470c6b0f-207c-4b89-9436-b4c2dad1f8ed None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 21 21:09:00.456836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:00.456836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:00.456836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:00.456836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:00.456836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:00.456836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:00.456836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:00.456836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:00.456836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:00.456836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:00.456836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:00.456836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:09:00.456836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:09:00.456836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:09:00.456836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:00.457865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:00.458796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:00.458796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:00.458796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:09:00.458796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:09:00.458796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:09:00.458796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:09:00.458796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:09:00.458796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:09:00.458796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 21 21:09:00.458796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:00.458796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 328/661] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Oct 21 21:09:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:00.463706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c6b0f-207c-4b89-9436-b4c2dad1f8ed None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:00.466980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:00.566493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d6fbd-f286-46a7-819a-ddd5c2f4cdf4 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:00.608516 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c6b0f-207c-4b89-9436-b4c2dad1f8ed None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:00.613680 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc3800-aa49-4dc6-a447-fd7290aeccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:00.614498 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc3800-aa49-4dc6-a447-fd7290aeccb7 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:00.614780 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc3800-aa49-4dc6-a447-fd7290aeccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:00.615047 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc3800-aa49-4dc6-a447-fd7290aeccb7 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:00.616118 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc3800-aa49-4dc6-a447-fd7290aeccb7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:00.616966 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc3800-aa49-4dc6-a447-fd7290aeccb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:00.617779 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc3800-aa49-4dc6-a447-fd7290aeccb7 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:00.624635 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 334/662] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Oct 21 21:09:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:00.630923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc3800-aa49-4dc6-a447-fd7290aeccb7 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:00.651144 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c3558-2fc5-4c94-ba73-7f4f0e0c6997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:00.651846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c3558-2fc5-4c94-ba73-7f4f0e0c6997 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:00.652290 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c3558-2fc5-4c94-ba73-7f4f0e0c6997 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:00.652619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c3558-2fc5-4c94-ba73-7f4f0e0c6997 None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:00.653656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c3558-2fc5-4c94-ba73-7f4f0e0c6997 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:00.653966 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c3558-2fc5-4c94-ba73-7f4f0e0c6997 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:00.655414 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c3558-2fc5-4c94-ba73-7f4f0e0c6997 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:00.681454 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 329/663] 10.222.0.13 () {56 vars in 1149 bytes} [Fri Oct 21 21:09:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:00.686618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c3558-2fc5-4c94-ba73-7f4f0e0c6997 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:00.724086 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:00.756083 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:00.766586 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc3800-aa49-4dc6-a447-fd7290aeccb7 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:00.797073 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c3558-2fc5-4c94-ba73-7f4f0e0c6997 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:00.802778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2bda7-1e2c-4358-a44d-92008314b554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:00.803692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2bda7-1e2c-4358-a44d-92008314b554 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:00.804185 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2bda7-1e2c-4358-a44d-92008314b554 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:00.804676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2bda7-1e2c-4358-a44d-92008314b554 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:00.805747 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2bda7-1e2c-4358-a44d-92008314b554 None admin] RBAC: Adding query filter params (interface=public, service_id=0c636b2b49a943b09db056e5cdfc1eee, region_id=RegionOne) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:00.806273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2bda7-1e2c-4358-a44d-92008314b554 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:00.807857 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2bda7-1e2c-4358-a44d-92008314b554 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:00.818463 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 335/664] 10.222.0.13 () {56 vars in 1251 bytes} [Fri Oct 21 21:09:00 2022] GET /identity/v3/endpoints?service_id=0c636b2b49a943b09db056e5cdfc1eee&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:09:00.819681 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2bda7-1e2c-4358-a44d-92008314b554 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:00.832480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9166dcc-fc7b-4275-89f1-caef9d4a8611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:00.833115 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9166dcc-fc7b-4275-89f1-caef9d4a8611 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:00.833420 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9166dcc-fc7b-4275-89f1-caef9d4a8611 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:00.833920 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9166dcc-fc7b-4275-89f1-caef9d4a8611 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:00.843899 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9166dcc-fc7b-4275-89f1-caef9d4a8611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:00.845372 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9166dcc-fc7b-4275-89f1-caef9d4a8611 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:00.860648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-d9166dcc-fc7b-4275-89f1-caef9d4a8611 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:00.860648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:00.860648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:00.860648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:00.860648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:00.860648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:00.860648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:00.860648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:00.860648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:00.860648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:00.860648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:00.860648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:00.860648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:09:00.860648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:09:00.860648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:09:00.860648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:09:00.862237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:00.862237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:00.862237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:00.862237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:00.862237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:00.862237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:00.862237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:00.862237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:00.862237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:00.862237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:00.862237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:00.862237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:00.862237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:00.862237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:00.862237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:00.862237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:00.862237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:09:00.863091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:09:00.863091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:00.863091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:00.866581 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 330/665] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:09:00.867515 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9166dcc-fc7b-4275-89f1-caef9d4a8611 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:00.937031 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2bda7-1e2c-4358-a44d-92008314b554 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:00.972167 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9166dcc-fc7b-4275-89f1-caef9d4a8611 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:00.973282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fce9a-6d23-401f-9cb9-e62c4d29c2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:00.974132 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fce9a-6d23-401f-9cb9-e62c4d29c2ab None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:00.974630 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fce9a-6d23-401f-9cb9-e62c4d29c2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:00.975092 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fce9a-6d23-401f-9cb9-e62c4d29c2ab None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:00.975916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08fce9a-6d23-401f-9cb9-e62c4d29c2ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:00.977484 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08fce9a-6d23-401f-9cb9-e62c4d29c2ab None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:00.978868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:00.992454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 336/666] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Oct 21 21:09:00 2022] GET /identity/v3/services => generated 1258 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:09:01.001031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:01.008016 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a2be-689e-4627-91e2-1558c5673412 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:01.008304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a2be-689e-4627-91e2-1558c5673412 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:01.008577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a2be-689e-4627-91e2-1558c5673412 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:01.011728 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 337/667] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:09:01.012211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e14220-bc19-4831-b908-716b4c02259f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:01.013318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e14220-bc19-4831-b908-716b4c02259f None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:01.013817 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e14220-bc19-4831-b908-716b4c02259f None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:01.014366 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e14220-bc19-4831-b908-716b4c02259f None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:01.015926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12734ba7-a206-4dc6-ae35-5e03d631d135 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:01.016389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12734ba7-a206-4dc6-ae35-5e03d631d135 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:01.016709 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e14220-bc19-4831-b908-716b4c02259f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:01.016965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12734ba7-a206-4dc6-ae35-5e03d631d135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:01.017768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e14220-bc19-4831-b908-716b4c02259f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:01.019325 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e14220-bc19-4831-b908-716b4c02259f None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:01.032708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 331/668] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:09:01.038655 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e14220-bc19-4831-b908-716b4c02259f None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:01.046221 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-12734ba7-a206-4dc6-ae35-5e03d631d135 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:01.049048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-12734ba7-a206-4dc6-ae35-5e03d631d135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:01.148985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 338/669] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:01 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:01.155053 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e14220-bc19-4831-b908-716b4c02259f None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:01.195880 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c22c15-6b9a-4786-8853-784c7ce7cd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:01.196882 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c22c15-6b9a-4786-8853-784c7ce7cd91 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:01.197421 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c22c15-6b9a-4786-8853-784c7ce7cd91 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:01.197692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c22c15-6b9a-4786-8853-784c7ce7cd91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:01.204657 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c22c15-6b9a-4786-8853-784c7ce7cd91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:01.206215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c22c15-6b9a-4786-8853-784c7ce7cd91 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:01.217535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-93c22c15-6b9a-4786-8853-784c7ce7cd91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:09:01.217535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:01.217535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:01.217535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:01.217535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:01.217535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:01.217535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:01.217535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:01.217535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:01.217535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:01.217535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:01.217535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:01.217535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:09:01.217535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:09:01.217535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:09:01.218480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:09:01.218480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:01.218480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:01.218480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:01.218480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:01.218480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:01.218480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:01.218480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:01.218480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:01.218480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:01.218480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:01.218480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:01.218480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:01.218480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:01.218480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:01.218480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:01.218480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:01.219391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:09:01.219391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:09:01.219391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:09:01.219391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:09:01.219391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:09:01.219391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:09:01.219391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:09:01.219391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:01.219391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 332/670] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:09:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:09:01.225165 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12734ba7-a206-4dc6-ae35-5e03d631d135 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:01.330721 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12734ba7-a206-4dc6-ae35-5e03d631d135 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:01.358616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:01.368445 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4efa73-b2c6-49aa-bdff-037debdc2869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:01.369141 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4efa73-b2c6-49aa-bdff-037debdc2869 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:01.369421 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4efa73-b2c6-49aa-bdff-037debdc2869 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:01.369694 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4efa73-b2c6-49aa-bdff-037debdc2869 None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:01.370469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4efa73-b2c6-49aa-bdff-037debdc2869 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:01.370777 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4efa73-b2c6-49aa-bdff-037debdc2869 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:01.372285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4efa73-b2c6-49aa-bdff-037debdc2869 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:01.386123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 339/671] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:09:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:01.392375 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c22c15-6b9a-4786-8853-784c7ce7cd91 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:01.417499 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4efa73-b2c6-49aa-bdff-037debdc2869 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:01.430209 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:01.514205 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c22c15-6b9a-4786-8853-784c7ce7cd91 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:01.530236 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4efa73-b2c6-49aa-bdff-037debdc2869 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:01.556230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6e47e-5c01-4ad1-89cc-8ee6cb121a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:01.556891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6e47e-5c01-4ad1-89cc-8ee6cb121a3c None admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:01.557170 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6e47e-5c01-4ad1-89cc-8ee6cb121a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:01.557447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6e47e-5c01-4ad1-89cc-8ee6cb121a3c None admin] PATH_INFO: `/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/2346003afe74477496045e9bc72173d5/roles/d47c18046c97412588d8a6ae24333070` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:01.560482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6e47e-5c01-4ad1-89cc-8ee6cb121a3c None admin] RBAC: Authorizing `identity:create_grant(project_id=8b2c33250ef84d79a4a4742fb4252156, user_id=2346003afe74477496045e9bc72173d5, role_id=d47c18046c97412588d8a6ae24333070)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:01.561923 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6e47e-5c01-4ad1-89cc-8ee6cb121a3c None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:01.572156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913472cc-80a9-4526-9af1-057667e54751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:01.573035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913472cc-80a9-4526-9af1-057667e54751 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:01.573354 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913472cc-80a9-4526-9af1-057667e54751 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:01.574065 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913472cc-80a9-4526-9af1-057667e54751 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:01.574736 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913472cc-80a9-4526-9af1-057667e54751 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:01.576407 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913472cc-80a9-4526-9af1-057667e54751 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:01.583681 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 333/672] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:09:01 2022] PUT /identity/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/2346003afe74477496045e9bc72173d5/roles/d47c18046c97412588d8a6ae24333070 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:09:01.598628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 340/673] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Oct 21 21:09:01 2022] POST /identity/v3/services => generated 251 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:09:02.714803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6ed0b-be42-49ac-85f7-98abda40f591 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:02.715370 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6ed0b-be42-49ac-85f7-98abda40f591 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:02.715734 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6ed0b-be42-49ac-85f7-98abda40f591 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:02.718424 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 334/674] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:02.731006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c8cb2-477b-4c8f-b083-34e0e79ed30c None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:02.731462 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c8cb2-477b-4c8f-b083-34e0e79ed30c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:02.731830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c8cb2-477b-4c8f-b083-34e0e79ed30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:02.760566 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-766c8cb2-477b-4c8f-b083-34e0e79ed30c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:02.762521 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-766c8cb2-477b-4c8f-b083-34e0e79ed30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:02.812499 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:02.843763 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 341/675] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:03.283233 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6243e-8032-451b-b822-d2df867772b6 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:03.283793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6243e-8032-451b-b822-d2df867772b6 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:03.284282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6243e-8032-451b-b822-d2df867772b6 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:03.287086 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 335/676] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:03.292227 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54941de8-6a05-49ea-9832-54642f59eafd None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:03.292779 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54941de8-6a05-49ea-9832-54642f59eafd None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:03.293290 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54941de8-6a05-49ea-9832-54642f59eafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:03.312004 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-54941de8-6a05-49ea-9832-54642f59eafd None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:03.313667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-54941de8-6a05-49ea-9832-54642f59eafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:03.332453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6243e-8032-451b-b822-d2df867772b6 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:03.395317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 342/677] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:03.442453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:03.452999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6243e-8032-451b-b822-d2df867772b6 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:03.490558 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37182e2-4985-49f4-b3d3-91e44a2867df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:03.491193 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37182e2-4985-49f4-b3d3-91e44a2867df None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:03.491469 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37182e2-4985-49f4-b3d3-91e44a2867df None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:03.491769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37182e2-4985-49f4-b3d3-91e44a2867df None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:03.492340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37182e2-4985-49f4-b3d3-91e44a2867df None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:03.493722 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37182e2-4985-49f4-b3d3-91e44a2867df None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:03.502615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-b37182e2-4985-49f4-b3d3-91e44a2867df None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 21:09:03.502615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:03.502615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:09:03.502615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:09:03.502615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:09:03.502615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:09:03.502615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:09:03.502615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:03.502615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 21:09:03.502615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:03.502615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:03.502615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:03.502615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:03.502615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:03.502615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:03.502615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:03.502615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:03.503619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:03.503619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:03.503619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:03.503619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:03.503619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:03.503619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:03.503619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:09:03.503619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:09:03.503619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:09:03.503619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:09:03.503619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:03.503619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:03.503619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:03.503619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:03.503619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:03.503619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:03.504656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:03.504656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:03.504656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:03.504656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:03.504656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:03.504656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:03.504656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:03.504656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:03.504656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:03.504656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:03.504656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:09:03.504656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:03.504656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 21:09:03.504656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:03.504656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 336/678] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Oct 21 21:09:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:09:03.512096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54941de8-6a05-49ea-9832-54642f59eafd None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:03.614746 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54941de8-6a05-49ea-9832-54642f59eafd None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:03.617573 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:03.656143 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521d026-cff2-4546-bf34-2122132167eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:03.656752 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521d026-cff2-4546-bf34-2122132167eb None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:03.657028 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521d026-cff2-4546-bf34-2122132167eb None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:03.657302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521d026-cff2-4546-bf34-2122132167eb None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:03.657959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b521d026-cff2-4546-bf34-2122132167eb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:03.658263 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b521d026-cff2-4546-bf34-2122132167eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:03.659637 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b521d026-cff2-4546-bf34-2122132167eb None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:03.664910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 343/679] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Oct 21 21:09:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:03.670718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37182e2-4985-49f4-b3d3-91e44a2867df None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:03.690797 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521d026-cff2-4546-bf34-2122132167eb None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:03.801926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521d026-cff2-4546-bf34-2122132167eb None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:03.815910 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37182e2-4985-49f4-b3d3-91e44a2867df None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:03.838792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db59bb-d4f6-4784-87ba-1863a95bd795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:03.839427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db59bb-d4f6-4784-87ba-1863a95bd795 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:03.839735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db59bb-d4f6-4784-87ba-1863a95bd795 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:03.840007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db59bb-d4f6-4784-87ba-1863a95bd795 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:03.840640 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db59bb-d4f6-4784-87ba-1863a95bd795 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:03.841974 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db59bb-d4f6-4784-87ba-1863a95bd795 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:03.850945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-22db59bb-d4f6-4784-87ba-1863a95bd795 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 21:09:03.850945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:03.850945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:09:03.850945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:09:03.850945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:09:03.850945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:09:03.850945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:09:03.850945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:03.850945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 21:09:03.850945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:03.850945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:03.850945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:03.850945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:03.850945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:03.850945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:03.850945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:03.852071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:03.852071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:03.852071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:03.852071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:03.852071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:03.852071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:03.852071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:03.852071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:09:03.852071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:09:03.852071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:09:03.852071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:09:03.852071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:03.852071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:03.852071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:03.852071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:03.852071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:03.852071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:03.853144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:03.853144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:03.853144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:03.853144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:03.853144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:03.853144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:03.853144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:03.853144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:03.853144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:03.853144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:03.853144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:09:03.853144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:03.853144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 21:09:03.853144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:03.854052 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ad47f-26bf-4573-9934-bb440c7e4210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:03.854052 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ad47f-26bf-4573-9934-bb440c7e4210 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:03.854271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ad47f-26bf-4573-9934-bb440c7e4210 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:03.854541 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ad47f-26bf-4573-9934-bb440c7e4210 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:03.855156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ad47f-26bf-4573-9934-bb440c7e4210 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:03.855452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ad47f-26bf-4573-9934-bb440c7e4210 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:03.856324 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 344/680] 10.222.0.13 () {56 vars in 1132 bytes} [Fri Oct 21 21:09:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:09:03.856836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ad47f-26bf-4573-9934-bb440c7e4210 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:03.857889 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4356f6-adae-470b-83e0-2c04e4d6659a None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:03.858309 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4356f6-adae-470b-83e0-2c04e4d6659a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:03.858586 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4356f6-adae-470b-83e0-2c04e4d6659a None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:03.859980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 345/681] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:03.865316 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4356f6-adae-470b-83e0-2c04e4d6659a None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:03.865569 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 337/682] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Oct 21 21:09:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:03.869827 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91867b-a51b-4031-92f1-b26e19e59285 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:03.870109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91867b-a51b-4031-92f1-b26e19e59285 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:03.870513 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91867b-a51b-4031-92f1-b26e19e59285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:03.890657 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-1a91867b-a51b-4031-92f1-b26e19e59285 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:03.892240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-1a91867b-a51b-4031-92f1-b26e19e59285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:03.900212 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:03.908471 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:03.964552 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 338/683] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:03.967766 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91867b-a51b-4031-92f1-b26e19e59285 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:03.981095 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4356f6-adae-470b-83e0-2c04e4d6659a None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:04.019347 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa3fa3-17df-463c-8c08-ed13c51255b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:04.020373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa3fa3-17df-463c-8c08-ed13c51255b5 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:04.020802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa3fa3-17df-463c-8c08-ed13c51255b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:04.021225 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa3fa3-17df-463c-8c08-ed13c51255b5 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:04.022246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa3fa3-17df-463c-8c08-ed13c51255b5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:04.022717 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa3fa3-17df-463c-8c08-ed13c51255b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:04.024411 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa3fa3-17df-463c-8c08-ed13c51255b5 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:04.030989 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 346/684] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Oct 21 21:09:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:04.038788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa3fa3-17df-463c-8c08-ed13c51255b5 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:04.099398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91867b-a51b-4031-92f1-b26e19e59285 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:04.136890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:04.144076 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa3fa3-17df-463c-8c08-ed13c51255b5 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:04.147465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1904cf2-f401-4036-be77-02f6c283032c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:04.148374 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1904cf2-f401-4036-be77-02f6c283032c None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:04.148873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1904cf2-f401-4036-be77-02f6c283032c None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:04.149399 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1904cf2-f401-4036-be77-02f6c283032c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:04.150311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1904cf2-f401-4036-be77-02f6c283032c None admin] RBAC: Adding query filter params (interface=public, service_id=3c7b8e2c63bf483b834dc3b585d51928, region_id=RegionOne) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:04.150757 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1904cf2-f401-4036-be77-02f6c283032c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:04.152668 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1904cf2-f401-4036-be77-02f6c283032c None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:04.160921 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 339/685] 10.222.0.13 () {56 vars in 1251 bytes} [Fri Oct 21 21:09:03 2022] GET /identity/v3/endpoints?service_id=3c7b8e2c63bf483b834dc3b585d51928&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:04.162499 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a5ba1-7aca-4b14-96c7-91697a362535 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:04.162945 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a5ba1-7aca-4b14-96c7-91697a362535 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:04.163354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a5ba1-7aca-4b14-96c7-91697a362535 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:04.165585 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 340/686] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:04.166426 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a5ba1-7aca-4b14-96c7-91697a362535 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:04.181949 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0be513-e741-45cc-a433-7410a9b6a8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:04.183189 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0be513-e741-45cc-a433-7410a9b6a8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:04.184289 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:04.185550 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0be513-e741-45cc-a433-7410a9b6a8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:04.185975 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0be513-e741-45cc-a433-7410a9b6a8b0 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:04.186637 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0be513-e741-45cc-a433-7410a9b6a8b0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:04.187114 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0be513-e741-45cc-a433-7410a9b6a8b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:04.188431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0be513-e741-45cc-a433-7410a9b6a8b0 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:04.193549 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 347/687] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Oct 21 21:09:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:04.194544 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dbb85-45ad-42e3-8e4c-9f778ab6f060 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:04.194826 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dbb85-45ad-42e3-8e4c-9f778ab6f060 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:04.195098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dbb85-45ad-42e3-8e4c-9f778ab6f060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:04.212788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-c15dbb85-45ad-42e3-8e4c-9f778ab6f060 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:04.214056 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c15dbb85-45ad-42e3-8e4c-9f778ab6f060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:04.275804 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a5ba1-7aca-4b14-96c7-91697a362535 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:04.281018 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 348/688] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:04.281716 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dbb85-45ad-42e3-8e4c-9f778ab6f060 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:04.312091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad31f7-e495-4cfe-afcb-249343a417de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:04.312808 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad31f7-e495-4cfe-afcb-249343a417de None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:04.313088 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad31f7-e495-4cfe-afcb-249343a417de None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:04.313363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad31f7-e495-4cfe-afcb-249343a417de None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:04.313975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad31f7-e495-4cfe-afcb-249343a417de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:04.315516 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad31f7-e495-4cfe-afcb-249343a417de None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:04.321571 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 341/689] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Oct 21 21:09:04 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:09:04.322423 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad31f7-e495-4cfe-afcb-249343a417de None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:04.398739 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:04.413567 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dbb85-45ad-42e3-8e4c-9f778ab6f060 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:04.447496 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:04.459818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a53938-f830-4c7a-8f10-bad4ba7d517b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:04.460700 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a53938-f830-4c7a-8f10-bad4ba7d517b None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:04.461135 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a53938-f830-4c7a-8f10-bad4ba7d517b None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:04.462212 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a53938-f830-4c7a-8f10-bad4ba7d517b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:04.463263 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a53938-f830-4c7a-8f10-bad4ba7d517b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:04.465267 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a53938-f830-4c7a-8f10-bad4ba7d517b None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:04.476933 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad31f7-e495-4cfe-afcb-249343a417de None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:04.503614 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 349/690] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:09:04 2022] POST /identity/v3/endpoints => generated 347 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:09:04.539053 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bf5b7-7268-455c-b071-91d48f2a6210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:04.539755 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bf5b7-7268-455c-b071-91d48f2a6210 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:04.540034 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bf5b7-7268-455c-b071-91d48f2a6210 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:04.540304 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bf5b7-7268-455c-b071-91d48f2a6210 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:04.540895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053bf5b7-7268-455c-b071-91d48f2a6210 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:04.542311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053bf5b7-7268-455c-b071-91d48f2a6210 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:04.552670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-053bf5b7-7268-455c-b071-91d48f2a6210 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 21:09:04.552670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:04.552670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:09:04.552670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:09:04.552670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:09:04.552670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:09:04.552670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:09:04.552670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:04.552670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 21:09:04.552670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:04.552670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:04.552670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:04.552670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:04.552670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:04.552670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:04.552670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:04.552670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:04.553776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:04.553776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:04.553776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:04.553776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:04.553776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:04.553776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:04.553776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:09:04.553776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:09:04.553776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:09:04.553776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:09:04.553776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:04.553776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:04.553776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:04.553776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:04.553776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:04.553776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:04.554639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:04.554639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:04.554639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:04.554639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:04.554639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:04.554639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:04.554639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:04.554639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:04.554639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:04.554639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:04.554639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:09:04.554639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:04.554639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 21:09:04.554639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:04.557421 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 342/691] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Oct 21 21:09:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:04.558944 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a53938-f830-4c7a-8f10-bad4ba7d517b None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:04.560597 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bf5b7-7268-455c-b071-91d48f2a6210 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:04.683481 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bf5b7-7268-455c-b071-91d48f2a6210 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:04.706890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a53938-f830-4c7a-8f10-bad4ba7d517b None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:04.736698 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15b335-7c03-4b2c-9e9b-301a39193230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:04.737957 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15b335-7c03-4b2c-9e9b-301a39193230 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:04.738903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15b335-7c03-4b2c-9e9b-301a39193230 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:04.739697 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15b335-7c03-4b2c-9e9b-301a39193230 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:04.740769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15b335-7c03-4b2c-9e9b-301a39193230 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:04.741317 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15b335-7c03-4b2c-9e9b-301a39193230 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:04.743887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15b335-7c03-4b2c-9e9b-301a39193230 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:04.749572 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba4dde-fac5-4f88-81d0-67cf89cdf06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:04.750210 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba4dde-fac5-4f88-81d0-67cf89cdf06f None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:04.750484 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba4dde-fac5-4f88-81d0-67cf89cdf06f None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:04.750749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba4dde-fac5-4f88-81d0-67cf89cdf06f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:04.754311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 343/692] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Oct 21 21:09:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:09:04.759793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15b335-7c03-4b2c-9e9b-301a39193230 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:04.763890 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:04.767755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba4dde-fac5-4f88-81d0-67cf89cdf06f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:09:04.773232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-ffba4dde-fac5-4f88-81d0-67cf89cdf06f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 21 21:09:04.773232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:04.773232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:04.773232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:04.773232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:04.773232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:04.773232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:04.773232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:04.773232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:04.773232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:04.773232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:04.773232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:04.773232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:09:04.773232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:09:04.773232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:09:04.773232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:09:04.774145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:09:04.774145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:09:04.774145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:09:04.774145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:09:04.774145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:04.774145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:04.774145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:04.774145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:04.774145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:04.774145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:04.774145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:04.774145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:04.774145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:04.774145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:04.774145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:04.774145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:04.774988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:04.774988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:04.774988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:04.774988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:04.774988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:09:04.774988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:09:04.774988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:09:04.774988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:09:04.774988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:09:04.774988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:09:04.774988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 21 21:09:04.774988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:04.774988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 350/693] 10.222.0.13 () {56 vars in 1120 bytes} [Fri Oct 21 21:09:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:04.781110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba4dde-fac5-4f88-81d0-67cf89cdf06f None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:04.792619 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:04.877436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15b335-7c03-4b2c-9e9b-301a39193230 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:04.914566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a863b-0c21-46aa-81bf-76da56f70243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:04.915340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a863b-0c21-46aa-81bf-76da56f70243 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:04.915752 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a863b-0c21-46aa-81bf-76da56f70243 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:04.916054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba4dde-fac5-4f88-81d0-67cf89cdf06f None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:04.916520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a863b-0c21-46aa-81bf-76da56f70243 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:04.916970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a863b-0c21-46aa-81bf-76da56f70243 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:04.917372 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a863b-0c21-46aa-81bf-76da56f70243 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:04.919018 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a863b-0c21-46aa-81bf-76da56f70243 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:04.929522 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 344/694] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Oct 21 21:09:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:09:04.930980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a863b-0c21-46aa-81bf-76da56f70243 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:04.956846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ea56d-ac98-4935-93a4-f77f134de150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:04.957619 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ea56d-ac98-4935-93a4-f77f134de150 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:04.957988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ea56d-ac98-4935-93a4-f77f134de150 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:04.958763 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ea56d-ac98-4935-93a4-f77f134de150 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:04.959222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ea56d-ac98-4935-93a4-f77f134de150 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:04.959655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ea56d-ac98-4935-93a4-f77f134de150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:04.961227 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ea56d-ac98-4935-93a4-f77f134de150 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:04.971040 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 351/695] 10.222.0.13 () {56 vars in 1115 bytes} [Fri Oct 21 21:09:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:04.979602 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ea56d-ac98-4935-93a4-f77f134de150 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:05.009917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:05.047392 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:05.050591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a863b-0c21-46aa-81bf-76da56f70243 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:05.092133 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe4462-2858-476c-9d4e-aa6f4ca56a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:05.092753 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe4462-2858-476c-9d4e-aa6f4ca56a94 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:05.093038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe4462-2858-476c-9d4e-aa6f4ca56a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:05.093553 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe4462-2858-476c-9d4e-aa6f4ca56a94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:05.094260 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe4462-2858-476c-9d4e-aa6f4ca56a94 None admin] RBAC: Adding query filter params (interface=public, service_id=9c2bf5748619427696a73f8c1703ab34, region_id=RegionOne) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:05.094567 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe4462-2858-476c-9d4e-aa6f4ca56a94 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:05.096486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe4462-2858-476c-9d4e-aa6f4ca56a94 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:05.096795 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ea56d-ac98-4935-93a4-f77f134de150 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:05.107511 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 345/696] 10.222.0.13 () {56 vars in 1251 bytes} [Fri Oct 21 21:09:04 2022] GET /identity/v3/endpoints?service_id=9c2bf5748619427696a73f8c1703ab34&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:05.109331 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe4462-2858-476c-9d4e-aa6f4ca56a94 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:05.135830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cba333-68f7-4694-ae7e-10e8077bcef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:05.136429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cba333-68f7-4694-ae7e-10e8077bcef6 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:05.136706 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cba333-68f7-4694-ae7e-10e8077bcef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:05.136977 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cba333-68f7-4694-ae7e-10e8077bcef6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:05.148048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cba333-68f7-4694-ae7e-10e8077bcef6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:05.149485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cba333-68f7-4694-ae7e-10e8077bcef6 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:05.164241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-96cba333-68f7-4694-ae7e-10e8077bcef6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:05.164241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:05.164241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:05.164241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:05.164241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:05.164241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:05.164241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:05.164241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:05.164241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:05.164241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:05.164241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:05.164241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:05.164241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:09:05.164241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:09:05.164241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:09:05.164241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:09:05.165197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:05.165197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:05.165197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:05.165197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:05.165197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:05.165197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:05.165197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:05.165197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:05.165197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:05.165197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:05.165197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:05.165197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:05.165197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:05.165197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:05.165197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:05.165197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:05.165197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:09:05.166669 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:09:05.166669 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:05.166669 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:05.166669 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 352/697] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:09:05.170744 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cba333-68f7-4694-ae7e-10e8077bcef6 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:05.226860 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe4462-2858-476c-9d4e-aa6f4ca56a94 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:05.257926 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:05.266186 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2894c-37f4-491d-83ae-661afc36a495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:05.267718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2894c-37f4-491d-83ae-661afc36a495 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:05.268248 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2894c-37f4-491d-83ae-661afc36a495 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:05.268744 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2894c-37f4-491d-83ae-661afc36a495 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:05.269719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2894c-37f4-491d-83ae-661afc36a495 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:05.271384 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2894c-37f4-491d-83ae-661afc36a495 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:05.272731 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cba333-68f7-4694-ae7e-10e8077bcef6 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:05.285295 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 346/698] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Oct 21 21:09:05 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:09:05.295812 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:05.312809 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f89547-ac83-4460-b132-0188cb632adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:05.313438 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f89547-ac83-4460-b132-0188cb632adf None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:05.313717 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f89547-ac83-4460-b132-0188cb632adf None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:05.314055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f89547-ac83-4460-b132-0188cb632adf None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:05.314713 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f89547-ac83-4460-b132-0188cb632adf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:05.315016 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f89547-ac83-4460-b132-0188cb632adf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:05.316452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f89547-ac83-4460-b132-0188cb632adf None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:05.330765 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 353/699] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:09:05.336032 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2894c-37f4-491d-83ae-661afc36a495 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:05.462202 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2894c-37f4-491d-83ae-661afc36a495 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:05.498685 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e5375-b9b0-40fb-b0d0-24038715deb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:05.499401 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e5375-b9b0-40fb-b0d0-24038715deb8 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:05.499805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e5375-b9b0-40fb-b0d0-24038715deb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:05.500197 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e5375-b9b0-40fb-b0d0-24038715deb8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:05.508176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e5375-b9b0-40fb-b0d0-24038715deb8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:05.509862 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e5375-b9b0-40fb-b0d0-24038715deb8 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:05.522076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-ac1e5375-b9b0-40fb-b0d0-24038715deb8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 21 21:09:05.522076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:05.522076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:05.522076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:05.522076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:05.522076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:05.522076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:05.522076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:05.522076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:05.522076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:05.522076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:05.522076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:05.522076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:09:05.522076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:09:05.522076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:09:05.522076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:05.523246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:05.524540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:05.524540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:05.524540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:09:05.524540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:09:05.524540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:09:05.524540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:09:05.524540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:09:05.524540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:09:05.524540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 21 21:09:05.524540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:05.526800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 347/700] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Oct 21 21:09:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:05.531696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f89547-ac83-4460-b132-0188cb632adf None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:05.637228 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f89547-ac83-4460-b132-0188cb632adf None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:05.673578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf36e0-8b2a-4430-bc12-98551262c11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:05.674187 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf36e0-8b2a-4430-bc12-98551262c11e None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:05.674520 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf36e0-8b2a-4430-bc12-98551262c11e None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:05.674792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf36e0-8b2a-4430-bc12-98551262c11e None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:05.675561 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf36e0-8b2a-4430-bc12-98551262c11e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:05.675887 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf36e0-8b2a-4430-bc12-98551262c11e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:05.677311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf36e0-8b2a-4430-bc12-98551262c11e None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:05.695767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 354/701] 10.222.0.13 () {56 vars in 1149 bytes} [Fri Oct 21 21:09:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:09:05.701448 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e5375-b9b0-40fb-b0d0-24038715deb8 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:05.768884 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:05.815222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e5375-b9b0-40fb-b0d0-24038715deb8 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:05.849687 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced75e7-2d0f-4ce1-8123-d6ec2289a60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:05.850301 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced75e7-2d0f-4ce1-8123-d6ec2289a60d None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:05.850581 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced75e7-2d0f-4ce1-8123-d6ec2289a60d None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:05.850857 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced75e7-2d0f-4ce1-8123-d6ec2289a60d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:05.919875 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced75e7-2d0f-4ce1-8123-d6ec2289a60d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:05.921331 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced75e7-2d0f-4ce1-8123-d6ec2289a60d None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:05.975786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:06.023443 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 348/702] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:09:05 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 21:09:06.028946 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf36e0-8b2a-4430-bc12-98551262c11e None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:06.070296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:06.133004 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf36e0-8b2a-4430-bc12-98551262c11e None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:06.168228 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0ac2f-d93e-4c52-adf9-aa5cd31f2b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:06.168860 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0ac2f-d93e-4c52-adf9-aa5cd31f2b49 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:06.169135 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0ac2f-d93e-4c52-adf9-aa5cd31f2b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:06.169406 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0ac2f-d93e-4c52-adf9-aa5cd31f2b49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:06.180228 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0ac2f-d93e-4c52-adf9-aa5cd31f2b49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:06.181662 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0ac2f-d93e-4c52-adf9-aa5cd31f2b49 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:06.196304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-0eb0ac2f-d93e-4c52-adf9-aa5cd31f2b49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:06.196304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:06.196304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:06.196304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:06.196304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:06.196304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:06.196304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:06.196304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:06.196304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:06.196304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:06.196304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:06.196304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:06.196304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:09:06.196304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:09:06.196304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:09:06.196304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:09:06.197254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:06.197254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:06.197254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:06.197254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:06.197254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:06.197254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:06.197254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:06.197254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:06.197254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:06.197254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:06.197254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:06.197254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:06.197254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:06.197254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:06.197254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:06.197254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:06.197254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:09:06.198132 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:09:06.198132 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:06.198132 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:06.198132 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 355/703] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:09:06.204977 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced75e7-2d0f-4ce1-8123-d6ec2289a60d None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:06.313526 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced75e7-2d0f-4ce1-8123-d6ec2289a60d None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:06.348422 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa50d4-cc62-4eb2-aaaf-daaf0ae19967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:06.349033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa50d4-cc62-4eb2-aaaf-daaf0ae19967 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:06.349308 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa50d4-cc62-4eb2-aaaf-daaf0ae19967 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:06.349577 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa50d4-cc62-4eb2-aaaf-daaf0ae19967 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:06.350212 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa50d4-cc62-4eb2-aaaf-daaf0ae19967 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:06.350510 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa50d4-cc62-4eb2-aaaf-daaf0ae19967 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:06.351917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa50d4-cc62-4eb2-aaaf-daaf0ae19967 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:06.363868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 349/704] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:06.369329 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0ac2f-d93e-4c52-adf9-aa5cd31f2b49 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:06.468380 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:06.479791 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0ac2f-d93e-4c52-adf9-aa5cd31f2b49 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:06.515502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eda35e-98de-46d3-bf92-7849b9a0345f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:06.516195 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eda35e-98de-46d3-bf92-7849b9a0345f None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:06.516473 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eda35e-98de-46d3-bf92-7849b9a0345f None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:06.516793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eda35e-98de-46d3-bf92-7849b9a0345f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:06.523739 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eda35e-98de-46d3-bf92-7849b9a0345f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:06.525186 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eda35e-98de-46d3-bf92-7849b9a0345f None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:06.536868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-d0eda35e-98de-46d3-bf92-7849b9a0345f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:09:06.536868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:06.536868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:06.536868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:06.536868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:06.536868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:06.536868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:06.536868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:06.536868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:06.536868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:06.536868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:06.536868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:06.536868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:09:06.536868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:09:06.536868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:09:06.537768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:09:06.537768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:06.537768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:06.537768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:06.537768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:06.537768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:06.537768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:06.537768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:06.537768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:06.537768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:06.537768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:06.537768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:06.537768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:06.537768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:06.537768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:06.537768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:06.537768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:06.538643 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:09:06.538643 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:09:06.538643 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:09:06.538643 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:09:06.538643 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:09:06.538643 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:09:06.538643 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:09:06.538643 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:06.538643 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 356/705] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:09:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:06.545349 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa50d4-cc62-4eb2-aaaf-daaf0ae19967 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:06.561845 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:06.579672 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5e6bc-6749-48b8-b880-7a46e9d535d2 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:06.579928 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5e6bc-6749-48b8-b880-7a46e9d535d2 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:06.580154 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5e6bc-6749-48b8-b880-7a46e9d535d2 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:06.581479 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 357/706] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:09:06.587918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92ab12-8672-4abe-978a-f2d7ad76012d None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:06.588203 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92ab12-8672-4abe-978a-f2d7ad76012d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:06.588477 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92ab12-8672-4abe-978a-f2d7ad76012d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:06.610909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-9f92ab12-8672-4abe-978a-f2d7ad76012d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:06.613008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9f92ab12-8672-4abe-978a-f2d7ad76012d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:06.658508 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa50d4-cc62-4eb2-aaaf-daaf0ae19967 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:06.699068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1724f-527f-45a3-a8fa-9aa706a9ffd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:06.699687 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1724f-527f-45a3-a8fa-9aa706a9ffd1 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:06.699971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1724f-527f-45a3-a8fa-9aa706a9ffd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:06.700243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1724f-527f-45a3-a8fa-9aa706a9ffd1 None admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:06.700930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1724f-527f-45a3-a8fa-9aa706a9ffd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:06.701238 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1724f-527f-45a3-a8fa-9aa706a9ffd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:06.702756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1724f-527f-45a3-a8fa-9aa706a9ffd1 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:06.707902 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 358/707] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 21:09:06.720868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 350/708] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:09:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:06.727292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92ab12-8672-4abe-978a-f2d7ad76012d None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:06.817426 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:06.837044 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92ab12-8672-4abe-978a-f2d7ad76012d None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:06.853588 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce74f4c-82e8-4a8a-af25-ab2202a31f8d None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:06.853947 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce74f4c-82e8-4a8a-af25-ab2202a31f8d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:06.854221 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce74f4c-82e8-4a8a-af25-ab2202a31f8d None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:06.855934 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 351/709] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:09:06.861388 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731f064-4a8c-417c-92ba-c4a1fcd1044d None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:06.861673 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731f064-4a8c-417c-92ba-c4a1fcd1044d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:06.861945 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731f064-4a8c-417c-92ba-c4a1fcd1044d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:06.872263 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d350ece-2221-427f-98d8-4f822888c565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:06.872869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d350ece-2221-427f-98d8-4f822888c565 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:06.873148 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d350ece-2221-427f-98d8-4f822888c565 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:06.873419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d350ece-2221-427f-98d8-4f822888c565 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:06.874142 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d350ece-2221-427f-98d8-4f822888c565 None admin] RBAC: Adding query filter params (role.id=d47c18046c97412588d8a6ae24333070, scope.project.id=8b2c33250ef84d79a4a4742fb4252156, user.id=2346003afe74477496045e9bc72173d5) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:06.874450 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d350ece-2221-427f-98d8-4f822888c565 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:06.875957 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d350ece-2221-427f-98d8-4f822888c565 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:06.880850 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-e731f064-4a8c-417c-92ba-c4a1fcd1044d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:06.882080 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e731f064-4a8c-417c-92ba-c4a1fcd1044d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:06.882256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 359/710] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:09:06 2022] GET /identity/v3/role_assignments?user.id=2346003afe74477496045e9bc72173d5&scope.project.id=8b2c33250ef84d79a4a4742fb4252156&role.id=d47c18046c97412588d8a6ae24333070 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:06.959308 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 352/711] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:06.982381 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d350ece-2221-427f-98d8-4f822888c565 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:07.090903 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d350ece-2221-427f-98d8-4f822888c565 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:07.124769 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6845ed1-fe4f-4a49-b8ce-23d5df8b3df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:07.125408 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6845ed1-fe4f-4a49-b8ce-23d5df8b3df3 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:07.125686 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6845ed1-fe4f-4a49-b8ce-23d5df8b3df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:07.125956 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6845ed1-fe4f-4a49-b8ce-23d5df8b3df3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:07.127070 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6845ed1-fe4f-4a49-b8ce-23d5df8b3df3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:07.128471 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6845ed1-fe4f-4a49-b8ce-23d5df8b3df3 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:07.137691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-e6845ed1-fe4f-4a49-b8ce-23d5df8b3df3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 21:09:07.137691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:07.137691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:09:07.137691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:09:07.137691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:09:07.137691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:09:07.137691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:09:07.137691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:07.137691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 21:09:07.137691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:07.137691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:07.137691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:07.137691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:07.137691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:07.137691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:07.137691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:07.137691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:07.140412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:07.140412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:07.140412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:07.140412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:07.140412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:07.140412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:07.140412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:09:07.140412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:09:07.140412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:09:07.140412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:09:07.140412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:07.140412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:07.140412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:07.140412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:07.140412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:07.140412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:07.142803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:07.142803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:07.142803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:07.142803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:07.142803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:07.142803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:07.142803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:07.142803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:07.142803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:07.142803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:07.142803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:09:07.142803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:07.142803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 21:09:07.142803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:07.145055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 360/712] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Oct 21 21:09:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:07.148711 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731f064-4a8c-417c-92ba-c4a1fcd1044d None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:07.160240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:07.249250 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731f064-4a8c-417c-92ba-c4a1fcd1044d None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:07.264162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6845ed1-fe4f-4a49-b8ce-23d5df8b3df3 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:07.277213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:07.289230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190c2a4-2a85-4818-bd8c-97a295b26f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:07.289963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190c2a4-2a85-4818-bd8c-97a295b26f9b None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:07.290246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190c2a4-2a85-4818-bd8c-97a295b26f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:07.290515 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190c2a4-2a85-4818-bd8c-97a295b26f9b None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:07.291139 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190c2a4-2a85-4818-bd8c-97a295b26f9b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:07.291443 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190c2a4-2a85-4818-bd8c-97a295b26f9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:07.292850 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190c2a4-2a85-4818-bd8c-97a295b26f9b None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:07.299617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 353/713] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Oct 21 21:09:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:07.303934 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190c2a4-2a85-4818-bd8c-97a295b26f9b None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:07.377461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6845ed1-fe4f-4a49-b8ce-23d5df8b3df3 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:07.381780 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:07.412591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6733768-75d8-4c0b-a634-e29f1726eb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:07.413222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6733768-75d8-4c0b-a634-e29f1726eb45 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:07.413559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6733768-75d8-4c0b-a634-e29f1726eb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:07.413849 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6733768-75d8-4c0b-a634-e29f1726eb45 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:07.414331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6733768-75d8-4c0b-a634-e29f1726eb45 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:07.415754 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6733768-75d8-4c0b-a634-e29f1726eb45 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:07.420742 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190c2a4-2a85-4818-bd8c-97a295b26f9b None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:07.424899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-f6733768-75d8-4c0b-a634-e29f1726eb45 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 21:09:07.424899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:07.424899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:09:07.424899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:09:07.424899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:09:07.424899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:09:07.424899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:09:07.424899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:07.424899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 21:09:07.424899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:07.424899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:07.424899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:07.424899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:07.424899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:07.424899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:07.424899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:07.424899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:07.425941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:07.425941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:07.425941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:07.425941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:07.425941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:07.425941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:07.425941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:09:07.425941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:09:07.425941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:09:07.425941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:09:07.425941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:07.425941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:07.425941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:07.425941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:07.425941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:07.425941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:07.426912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:07.426912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:07.426912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:07.426912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:07.426912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:07.426912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:07.426912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:07.426912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:07.426912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:07.426912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:07.426912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:09:07.426912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:07.426912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 21:09:07.426912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:07.426912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 361/714] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:09:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:09:07.435428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6733768-75d8-4c0b-a634-e29f1726eb45 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:07.457880 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef0f63-3b27-4e20-87af-833d0bfa323e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:07.458499 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef0f63-3b27-4e20-87af-833d0bfa323e None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:07.458786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef0f63-3b27-4e20-87af-833d0bfa323e None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:07.459061 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef0f63-3b27-4e20-87af-833d0bfa323e None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:07.459717 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef0f63-3b27-4e20-87af-833d0bfa323e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:07.460023 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef0f63-3b27-4e20-87af-833d0bfa323e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:07.461420 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef0f63-3b27-4e20-87af-833d0bfa323e None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:07.471823 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 354/715] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Oct 21 21:09:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:07.472878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef0f63-3b27-4e20-87af-833d0bfa323e None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:07.525123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:07.578261 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6733768-75d8-4c0b-a634-e29f1726eb45 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:07.615762 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef0f63-3b27-4e20-87af-833d0bfa323e None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:07.619866 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670721c-3ad1-42b9-8973-4473e6e75bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:07.620716 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670721c-3ad1-42b9-8973-4473e6e75bfa None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:07.621191 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670721c-3ad1-42b9-8973-4473e6e75bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:07.621686 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670721c-3ad1-42b9-8973-4473e6e75bfa None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:07.622504 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670721c-3ad1-42b9-8973-4473e6e75bfa None admin] RBAC: Adding query filter params (name=compute) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:07.623020 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670721c-3ad1-42b9-8973-4473e6e75bfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:07.624896 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670721c-3ad1-42b9-8973-4473e6e75bfa None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:07.635233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 362/716] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Oct 21 21:09:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:09:07.636438 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670721c-3ad1-42b9-8973-4473e6e75bfa None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:07.650823 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:07.657142 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab102f-4a6b-4937-aea9-5b4bdf1c26ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:07.657838 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab102f-4a6b-4937-aea9-5b4bdf1c26ba None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:07.658119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab102f-4a6b-4937-aea9-5b4bdf1c26ba None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:07.658392 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab102f-4a6b-4937-aea9-5b4bdf1c26ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:07.659070 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab102f-4a6b-4937-aea9-5b4bdf1c26ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:07.660500 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab102f-4a6b-4937-aea9-5b4bdf1c26ba None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:07.687596 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 355/717] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:09:07 2022] POST /identity/v3/endpoints => generated 349 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:09:07.687596 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61fccf-b4bc-4222-a6cd-d90762b31d65 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:07.687596 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61fccf-b4bc-4222-a6cd-d90762b31d65 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:07.687596 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61fccf-b4bc-4222-a6cd-d90762b31d65 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:07.687596 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 356/718] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:07.690952 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c122d95-61d0-4734-954f-339c8836994e None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:07.691247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c122d95-61d0-4734-954f-339c8836994e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:07.691517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c122d95-61d0-4734-954f-339c8836994e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:07.717094 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-2c122d95-61d0-4734-954f-339c8836994e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:07.718616 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2c122d95-61d0-4734-954f-339c8836994e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:07.775172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670721c-3ad1-42b9-8973-4473e6e75bfa None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:07.816622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:07.824714 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 357/719] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 21:09:07.829893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287f0c1-5a6f-4fdb-9c9f-53c20da1a559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:07.831014 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287f0c1-5a6f-4fdb-9c9f-53c20da1a559 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:07.831509 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287f0c1-5a6f-4fdb-9c9f-53c20da1a559 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:07.832064 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287f0c1-5a6f-4fdb-9c9f-53c20da1a559 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:07.832971 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287f0c1-5a6f-4fdb-9c9f-53c20da1a559 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:07.833518 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287f0c1-5a6f-4fdb-9c9f-53c20da1a559 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:07.835323 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287f0c1-5a6f-4fdb-9c9f-53c20da1a559 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:07.841601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 363/720] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Oct 21 21:09:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:08.095726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c122d95-61d0-4734-954f-339c8836994e None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:08.198195 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c122d95-61d0-4734-954f-339c8836994e None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:08.213211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:08.234206 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01787e-f304-4fcb-9ecc-b5fbc8582bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:08.235336 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01787e-f304-4fcb-9ecc-b5fbc8582bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:08.236147 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01787e-f304-4fcb-9ecc-b5fbc8582bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:08.236483 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01787e-f304-4fcb-9ecc-b5fbc8582bf8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:08.237065 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01787e-f304-4fcb-9ecc-b5fbc8582bf8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:08.238446 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01787e-f304-4fcb-9ecc-b5fbc8582bf8 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:08.247826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-6f01787e-f304-4fcb-9ecc-b5fbc8582bf8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 21:09:08.247826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:08.247826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:09:08.247826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:09:08.247826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:09:08.247826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:09:08.247826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:09:08.247826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:08.247826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 21:09:08.247826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:08.247826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:08.247826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:08.247826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:08.247826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:08.247826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:08.247826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:08.247826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:08.249168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:08.249168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:08.249168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:08.249168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:08.249168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:08.249168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:08.249168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:09:08.249168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:09:08.249168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:09:08.249168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:09:08.249168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:08.249168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:08.249168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:08.249168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:08.249168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:08.249168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:08.250267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:08.250267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:08.250267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:08.250267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:08.250267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:08.250267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:08.250267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:08.250267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:08.250267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:08.250267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:08.250267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:09:08.250267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:08.250267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 21:09:08.250267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:08.250267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 358/721] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Oct 21 21:09:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:08.259598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287f0c1-5a6f-4fdb-9c9f-53c20da1a559 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:08.360413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287f0c1-5a6f-4fdb-9c9f-53c20da1a559 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:08.395405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c8fc1-d202-4bcd-a564-9b26681d15a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:08.396045 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c8fc1-d202-4bcd-a564-9b26681d15a4 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:08.396327 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c8fc1-d202-4bcd-a564-9b26681d15a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:08.396602 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c8fc1-d202-4bcd-a564-9b26681d15a4 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:08.397225 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c8fc1-d202-4bcd-a564-9b26681d15a4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:08.397531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c8fc1-d202-4bcd-a564-9b26681d15a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:08.398914 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c8fc1-d202-4bcd-a564-9b26681d15a4 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:08.407698 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 364/722] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Oct 21 21:09:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:08.409708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01787e-f304-4fcb-9ecc-b5fbc8582bf8 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:08.510737 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01787e-f304-4fcb-9ecc-b5fbc8582bf8 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:08.545794 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16a6b7-7138-4ab6-bcc9-e2da0b06771b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:08.546464 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16a6b7-7138-4ab6-bcc9-e2da0b06771b None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:08.546744 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16a6b7-7138-4ab6-bcc9-e2da0b06771b None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:08.547014 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16a6b7-7138-4ab6-bcc9-e2da0b06771b None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:08.547708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16a6b7-7138-4ab6-bcc9-e2da0b06771b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:08.548012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16a6b7-7138-4ab6-bcc9-e2da0b06771b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:08.549974 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16a6b7-7138-4ab6-bcc9-e2da0b06771b None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:08.559687 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 359/723] 10.222.0.13 () {56 vars in 1114 bytes} [Fri Oct 21 21:09:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:08.561415 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c8fc1-d202-4bcd-a564-9b26681d15a4 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:08.623685 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:08.668279 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c8fc1-d202-4bcd-a564-9b26681d15a4 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:08.703416 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd4410-1090-49c8-82c0-0622e62da034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:08.704080 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd4410-1090-49c8-82c0-0622e62da034 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:08.704691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd4410-1090-49c8-82c0-0622e62da034 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:08.704968 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd4410-1090-49c8-82c0-0622e62da034 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:08.705656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd4410-1090-49c8-82c0-0622e62da034 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:08.707176 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd4410-1090-49c8-82c0-0622e62da034 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:08.740367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 365/724] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:09:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:09:09.114596 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c1f00-eb55-4881-b795-3e5cd74db4b4 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:09.115125 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c1f00-eb55-4881-b795-3e5cd74db4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:09.115125 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c1f00-eb55-4881-b795-3e5cd74db4b4 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:09.116795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 360/725] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:09.122532 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99641585-7cea-4d4c-b508-9d9798088586 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:09.122822 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99641585-7cea-4d4c-b508-9d9798088586 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:09.123098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99641585-7cea-4d4c-b508-9d9798088586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:09.142926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-99641585-7cea-4d4c-b508-9d9798088586 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:09.144163 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-99641585-7cea-4d4c-b508-9d9798088586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:09.217736 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 366/726] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:09.487824 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c1f00-eb55-4881-b795-3e5cd74db4b4 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:09.531987 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:09.590640 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c1f00-eb55-4881-b795-3e5cd74db4b4 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:09.624952 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8822d1-bf59-4af0-a05f-d9143b08c9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:09.625590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8822d1-bf59-4af0-a05f-d9143b08c9ae None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:09.625867 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8822d1-bf59-4af0-a05f-d9143b08c9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:09.626678 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8822d1-bf59-4af0-a05f-d9143b08c9ae None admin] PATH_INFO: `/v3/services/image` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:09.627254 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8822d1-bf59-4af0-a05f-d9143b08c9ae None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:09.628649 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8822d1-bf59-4af0-a05f-d9143b08c9ae None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:09.637417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-ec8822d1-bf59-4af0-a05f-d9143b08c9ae None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 21:09:09.637417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:09.637417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:09:09.637417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:09:09.637417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:09:09.637417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:09:09.637417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:09:09.637417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:09.637417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 21:09:09.637417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:09.637417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:09.637417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:09.637417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:09.637417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:09.637417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:09.637417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:09.637417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:09.638736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:09.638736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:09.638736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:09.638736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:09.638736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:09.638736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:09.638736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:09:09.638736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:09:09.638736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:09:09.638736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:09:09.638736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:09.638736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:09.638736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:09.638736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:09.638736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:09.638736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:09.639592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:09.639592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:09.639592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:09.639592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:09.639592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:09.639592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:09.639592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:09.639592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:09.639592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:09.639592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:09.639592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:09:09.639592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:09.639592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 21:09:09.639592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:09.639592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 361/727] 10.222.0.13 () {56 vars in 1105 bytes} [Fri Oct 21 21:09:09 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:09.646933 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99641585-7cea-4d4c-b508-9d9798088586 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:09.693352 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:09.755942 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99641585-7cea-4d4c-b508-9d9798088586 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:09.790329 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca76cc-47e1-4424-8c75-de8f2d65c798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:09.790946 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca76cc-47e1-4424-8c75-de8f2d65c798 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:09.791222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca76cc-47e1-4424-8c75-de8f2d65c798 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:09.791490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca76cc-47e1-4424-8c75-de8f2d65c798 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:09.792139 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca76cc-47e1-4424-8c75-de8f2d65c798 None admin] RBAC: Adding query filter params (name=image) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:09.792442 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca76cc-47e1-4424-8c75-de8f2d65c798 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:09.793803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca76cc-47e1-4424-8c75-de8f2d65c798 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:09.799234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 367/728] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:09.804557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8822d1-bf59-4af0-a05f-d9143b08c9ae None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:09.904019 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8822d1-bf59-4af0-a05f-d9143b08c9ae None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:09.912003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:09.939841 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db779a-72ab-483f-b349-547d6ef75a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:09.940458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db779a-72ab-483f-b349-547d6ef75a82 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:09.940734 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db779a-72ab-483f-b349-547d6ef75a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:09.941009 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db779a-72ab-483f-b349-547d6ef75a82 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:09.941639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db779a-72ab-483f-b349-547d6ef75a82 None admin] RBAC: Adding query filter params (type=image) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:09.941939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db779a-72ab-483f-b349-547d6ef75a82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:09.943350 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db779a-72ab-483f-b349-547d6ef75a82 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:09.948997 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 362/729] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:09 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:10.078540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4d972-aad4-461c-af22-c9bdd8c20b1c None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:10.078719 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4d972-aad4-461c-af22-c9bdd8c20b1c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:10.079003 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4d972-aad4-461c-af22-c9bdd8c20b1c None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:10.081272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 368/730] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:10.087107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9767f-0aa9-4f29-b316-17c7bdaf2783 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:10.087422 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9767f-0aa9-4f29-b316-17c7bdaf2783 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:10.087705 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9767f-0aa9-4f29-b316-17c7bdaf2783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:10.111101 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-dba9767f-0aa9-4f29-b316-17c7bdaf2783 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:10.112352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-dba9767f-0aa9-4f29-b316-17c7bdaf2783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:10.118801 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cddaa-04c1-4b8a-ac07-40095cfde4b4 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:10.119408 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cddaa-04c1-4b8a-ac07-40095cfde4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:10.119923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cddaa-04c1-4b8a-ac07-40095cfde4b4 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:10.121774 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 369/731] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:10.128024 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd147b7a-cf94-47ea-87a5-2e1de29ac7bf None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:10.128607 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd147b7a-cf94-47ea-87a5-2e1de29ac7bf None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:10.129085 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd147b7a-cf94-47ea-87a5-2e1de29ac7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:10.151050 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-fd147b7a-cf94-47ea-87a5-2e1de29ac7bf None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:10.152628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-fd147b7a-cf94-47ea-87a5-2e1de29ac7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:10.187857 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 363/732] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:10.239417 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 370/733] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:10.494818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9767f-0aa9-4f29-b316-17c7bdaf2783 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:10.557081 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd147b7a-cf94-47ea-87a5-2e1de29ac7bf None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:10.612353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:10.656049 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:10.660383 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9767f-0aa9-4f29-b316-17c7bdaf2783 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:10.696919 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f60b77-7d72-4df8-88c2-5b067c403135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:10.697937 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f60b77-7d72-4df8-88c2-5b067c403135 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:10.698422 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f60b77-7d72-4df8-88c2-5b067c403135 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:10.698907 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f60b77-7d72-4df8-88c2-5b067c403135 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:10.699732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f60b77-7d72-4df8-88c2-5b067c403135 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:10.701331 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f60b77-7d72-4df8-88c2-5b067c403135 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:10.711230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-a9f60b77-7d72-4df8-88c2-5b067c403135 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 21:09:10.711230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:10.711230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:09:10.711230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:09:10.711230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:09:10.711230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:09:10.711230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:09:10.711230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:10.711230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 21:09:10.711230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:10.711230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:10.711230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:10.711230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:10.711230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:10.711230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:10.711230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:10.711230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:10.712282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:10.712282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:10.712282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:10.712282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:10.712282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:10.712282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:10.712282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:09:10.712282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:09:10.712282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:09:10.712282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:09:10.712282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:10.712282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:10.712282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:10.712282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:10.712282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:10.712282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:10.713130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:10.713130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:10.713130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:10.713130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:10.713130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:10.713130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:10.713130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:10.713130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:10.713130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:10.713130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:10.713130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:09:10.713130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:10.713130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 21:09:10.713130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:10.716010 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 364/734] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Oct 21 21:09:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:10.729024 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f60b77-7d72-4df8-88c2-5b067c403135 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:10.753347 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd147b7a-cf94-47ea-87a5-2e1de29ac7bf None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:10.822645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278aadc-4914-405a-a4f3-5fe53869f22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:10.823681 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278aadc-4914-405a-a4f3-5fe53869f22e None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:10.824171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278aadc-4914-405a-a4f3-5fe53869f22e None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:10.824634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278aadc-4914-405a-a4f3-5fe53869f22e None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:10.825721 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278aadc-4914-405a-a4f3-5fe53869f22e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:10.828049 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278aadc-4914-405a-a4f3-5fe53869f22e None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:10.847970 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 371/735] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Oct 21 21:09:10 2022] POST /identity/v3/services => generated 242 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:09:10.906358 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f60b77-7d72-4df8-88c2-5b067c403135 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:10.953801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26600a73-199c-4ed9-8f04-6a35dd480fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:10.954496 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26600a73-199c-4ed9-8f04-6a35dd480fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:10.954781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26600a73-199c-4ed9-8f04-6a35dd480fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:10.955053 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26600a73-199c-4ed9-8f04-6a35dd480fd7 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:10.955719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26600a73-199c-4ed9-8f04-6a35dd480fd7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:10.956021 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26600a73-199c-4ed9-8f04-6a35dd480fd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:10.957534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26600a73-199c-4ed9-8f04-6a35dd480fd7 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:10.963502 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 365/736] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Oct 21 21:09:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:10.969208 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278aadc-4914-405a-a4f3-5fe53869f22e None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:11.078630 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:11.085502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278aadc-4914-405a-a4f3-5fe53869f22e None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:11.121442 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730b3d1-0ec7-4df0-a9d0-e0f888ec2b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:11.122055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730b3d1-0ec7-4df0-a9d0-e0f888ec2b1c None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:11.122332 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730b3d1-0ec7-4df0-a9d0-e0f888ec2b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:11.122603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730b3d1-0ec7-4df0-a9d0-e0f888ec2b1c None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:11.123239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730b3d1-0ec7-4df0-a9d0-e0f888ec2b1c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:11.123542 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730b3d1-0ec7-4df0-a9d0-e0f888ec2b1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:11.124972 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730b3d1-0ec7-4df0-a9d0-e0f888ec2b1c None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:11.131031 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 372/737] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Oct 21 21:09:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:11.136758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730b3d1-0ec7-4df0-a9d0-e0f888ec2b1c None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:11.137408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d897a0-2e5d-4a3b-86a4-963d4467181e None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:11.138169 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d897a0-2e5d-4a3b-86a4-963d4467181e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:11.138637 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d897a0-2e5d-4a3b-86a4-963d4467181e None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:11.140430 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 366/738] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:11.145932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e491ef4-98df-4e02-837b-3f9ef42c1a01 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:11.146464 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e491ef4-98df-4e02-837b-3f9ef42c1a01 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:11.146793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e491ef4-98df-4e02-837b-3f9ef42c1a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:11.165473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-4e491ef4-98df-4e02-837b-3f9ef42c1a01 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:11.166982 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-4e491ef4-98df-4e02-837b-3f9ef42c1a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:11.185842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:11.240481 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 367/739] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 21:09:11.260789 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730b3d1-0ec7-4df0-a9d0-e0f888ec2b1c None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:11.296664 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1d428-b206-4041-a4b2-8efceaa7d874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:11.297279 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1d428-b206-4041-a4b2-8efceaa7d874 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:11.297559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1d428-b206-4041-a4b2-8efceaa7d874 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:11.297835 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1d428-b206-4041-a4b2-8efceaa7d874 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:11.299062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1d428-b206-4041-a4b2-8efceaa7d874 None admin] RBAC: Adding query filter params (interface=admin, service_id=3c7b8e2c63bf483b834dc3b585d51928, region_id=RegionOne) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:11.299371 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1d428-b206-4041-a4b2-8efceaa7d874 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:11.300826 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1d428-b206-4041-a4b2-8efceaa7d874 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:11.306143 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 373/740] 10.222.0.13 () {56 vars in 1249 bytes} [Fri Oct 21 21:09:11 2022] GET /identity/v3/endpoints?service_id=3c7b8e2c63bf483b834dc3b585d51928&interface=admin®ion_id=RegionOne => generated 193 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:11.312018 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e491ef4-98df-4e02-837b-3f9ef42c1a01 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:11.416736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e491ef4-98df-4e02-837b-3f9ef42c1a01 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:11.452195 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378593ac-057d-4804-b213-f0bd4fcbdf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:11.452807 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378593ac-057d-4804-b213-f0bd4fcbdf81 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:11.453086 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378593ac-057d-4804-b213-f0bd4fcbdf81 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:11.453361 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378593ac-057d-4804-b213-f0bd4fcbdf81 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:11.453957 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378593ac-057d-4804-b213-f0bd4fcbdf81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:11.455890 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378593ac-057d-4804-b213-f0bd4fcbdf81 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:11.462474 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 368/741] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Oct 21 21:09:11 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:09:11.522687 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1d428-b206-4041-a4b2-8efceaa7d874 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:11.557102 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:11.641336 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1d428-b206-4041-a4b2-8efceaa7d874 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:11.676590 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80a231-a63c-4fe8-9214-80d9b0069dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:11.677524 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80a231-a63c-4fe8-9214-80d9b0069dd8 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:11.677998 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80a231-a63c-4fe8-9214-80d9b0069dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:11.678597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80a231-a63c-4fe8-9214-80d9b0069dd8 None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:11.679440 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80a231-a63c-4fe8-9214-80d9b0069dd8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:11.681066 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80a231-a63c-4fe8-9214-80d9b0069dd8 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:11.683876 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: WARNING keystone.server.flask.common [None req-bd80a231-a63c-4fe8-9214-80d9b0069dd8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 21 21:09:11.704207 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.sql.core [None req-bd80a231-a63c-4fe8-9214-80d9b0069dd8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 21 21:09:11.704207 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 21 21:09:11.704207 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [parameters: {'id': '893bcd8e903e4432a09cb7fb1bae1abe', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 21 21:09:11.704207 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81305) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 21 21:09:11.712697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-bd80a231-a63c-4fe8-9214-80d9b0069dd8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 893bcd8e903e4432a09cb7fb1bae1abe.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 893bcd8e903e4432a09cb7fb1bae1abe. Oct 21 21:09:11.712697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:11.712697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 21:09:11.712697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 21:09:11.712697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 21:09:11.712697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 21:09:11.712697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 21:09:11.712697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 21:09:11.712697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 21:09:11.712697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application conn.query(q) Oct 21 21:09:11.712697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 21:09:11.712697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 21:09:11.712697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application result.read() Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 21 21:09:11.713467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 21 21:09:11.714388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 21:09:11.714388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application next(self.gen) Oct 21 21:09:11.714388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 21 21:09:11.714388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application yield resource Oct 21 21:09:11.714388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 21:09:11.714388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application next(self.gen) Oct 21 21:09:11.714388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 21 21:09:11.714388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self.session.rollback() Oct 21 21:09:11.714388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 21 21:09:11.714388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self.force_reraise() Oct 21 21:09:11.714388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 21 21:09:11.714388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise self.value Oct 21 21:09:11.714388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 21 21:09:11.714388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 21 21:09:11.714388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 21 21:09:11.714388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application session.commit() Oct 21 21:09:11.714388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 21 21:09:11.715337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 21 21:09:11.715337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 21 21:09:11.715337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._prepare_impl() Oct 21 21:09:11.715337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 21 21:09:11.715337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self.session.flush() Oct 21 21:09:11.715337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 21 21:09:11.715337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._flush(objects) Oct 21 21:09:11.715337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 21 21:09:11.715337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 21 21:09:11.715337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 21 21:09:11.715337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application compat.raise_( Oct 21 21:09:11.715337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 21:09:11.715337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception Oct 21 21:09:11.715337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 21 21:09:11.715337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application flush_context.execute() Oct 21 21:09:11.715337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 21 21:09:11.715337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rec.execute(self) Oct 21 21:09:11.716959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 21 21:09:11.716959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 21 21:09:11.716959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 21 21:09:11.716959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 21 21:09:11.716959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 21 21:09:11.716959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 21 21:09:11.716959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 21 21:09:11.716959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 21 21:09:11.716959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 21 21:09:11.716959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 21 21:09:11.716959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 21 21:09:11.716959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 21 21:09:11.716959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 21 21:09:11.716959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 21 21:09:11.716959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 21 21:09:11.718793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 21 21:09:11.718793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 21:09:11.718793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception Oct 21 21:09:11.718793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 21:09:11.718793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 21:09:11.718793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 21:09:11.718793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 21:09:11.718793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 21:09:11.718793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 21:09:11.718793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 21:09:11.718793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application conn.query(q) Oct 21 21:09:11.718793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 21:09:11.718793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 21:09:11.718793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 21:09:11.718793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application result.read() Oct 21 21:09:11.718793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 21:09:11.718793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 21:09:11.721037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 21:09:11.721037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 21:09:11.721037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 21:09:11.721037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 21:09:11.721037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 21:09:11.721037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 21:09:11.721037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 21 21:09:11.721037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 21 21:09:11.721037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [parameters: {'id': '893bcd8e903e4432a09cb7fb1bae1abe', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 21 21:09:11.721037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 21 21:09:11.721037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:11.721037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:11.721037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:11.721037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:11.721037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 21 21:09:11.722867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 21 21:09:11.722867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 21 21:09:11.722867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 21 21:09:11.722867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 21 21:09:11.722867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:11.722867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:11.722867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:11.722867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:11.722867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:11.722867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:11.722867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:11.722867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:11.722867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:11.722867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:11.722867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:11.722867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:11.722867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:11.724996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:11.724996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 21 21:09:11.724996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 21 21:09:11.724996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:11.724996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:11.724996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 21 21:09:11.724996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 21 21:09:11.724996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 893bcd8e903e4432a09cb7fb1bae1abe. Oct 21 21:09:11.724996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:11.731642 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 374/742] 10.222.0.13 () {60 vars in 1140 bytes} [Fri Oct 21 21:09:11 2022] POST /identity/v3/projects => generated 262 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:11.734600 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378593ac-057d-4804-b213-f0bd4fcbdf81 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:11.783958 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:11.837581 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378593ac-057d-4804-b213-f0bd4fcbdf81 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:11.875132 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2cfd5-80d7-4430-9118-63d40c7b4a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:11.875926 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2cfd5-80d7-4430-9118-63d40c7b4a0b None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:11.876911 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2cfd5-80d7-4430-9118-63d40c7b4a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:11.877354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2cfd5-80d7-4430-9118-63d40c7b4a0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:11.884385 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2cfd5-80d7-4430-9118-63d40c7b4a0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:11.885829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2cfd5-80d7-4430-9118-63d40c7b4a0b None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:11.897137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-a3f2cfd5-80d7-4430-9118-63d40c7b4a0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:09:11.897137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:11.897137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:11.897137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:11.897137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:11.897137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:11.897137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:11.897137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:11.897137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:11.897137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:11.897137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:11.897137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:11.897137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:09:11.897137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:09:11.897137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:09:11.899159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:09:11.899159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:11.899159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:11.899159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:11.899159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:11.899159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:11.899159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:11.899159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:11.899159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:11.899159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:11.899159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:11.899159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:11.899159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:11.899159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:11.899159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:11.899159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:11.899159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:11.901247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:09:11.901247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:09:11.901247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:09:11.901247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:09:11.901247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:09:11.901247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:09:11.901247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:09:11.901247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:11.901247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 369/743] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:09:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:11.908992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80a231-a63c-4fe8-9214-80d9b0069dd8 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:11.997615 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:12.019493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80a231-a63c-4fe8-9214-80d9b0069dd8 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:12.053934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910dc75-ef34-4a9f-b87a-eb2aaaa5c3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:12.054912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910dc75-ef34-4a9f-b87a-eb2aaaa5c3de None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:12.055388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910dc75-ef34-4a9f-b87a-eb2aaaa5c3de None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:12.055918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910dc75-ef34-4a9f-b87a-eb2aaaa5c3de None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:12.056767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6910dc75-ef34-4a9f-b87a-eb2aaaa5c3de None admin] RBAC: Adding query filter params (name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:12.057261 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6910dc75-ef34-4a9f-b87a-eb2aaaa5c3de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:12.059142 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6910dc75-ef34-4a9f-b87a-eb2aaaa5c3de None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:12.080343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 375/744] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Oct 21 21:09:11 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:12.222566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e3fb9-cf92-4351-b482-a4a225e37812 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:12.222960 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e3fb9-cf92-4351-b482-a4a225e37812 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:12.223134 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e3fb9-cf92-4351-b482-a4a225e37812 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:12.224876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 370/745] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:12.230968 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0bac1-d4ad-45d4-8e7f-7444c231ac3e None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:12.231272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0bac1-d4ad-45d4-8e7f-7444c231ac3e None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:12.231584 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0bac1-d4ad-45d4-8e7f-7444c231ac3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:12.251684 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-46e0bac1-d4ad-45d4-8e7f-7444c231ac3e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:12.253145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-46e0bac1-d4ad-45d4-8e7f-7444c231ac3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:12.327632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 376/746] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:12.593417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e3fb9-cf92-4351-b482-a4a225e37812 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:12.673797 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:12.697233 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e3fb9-cf92-4351-b482-a4a225e37812 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:12.739616 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6b176-cbdf-4b2e-addc-d979008085b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:12.739616 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6b176-cbdf-4b2e-addc-d979008085b8 None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:12.739616 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6b176-cbdf-4b2e-addc-d979008085b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:12.739616 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6b176-cbdf-4b2e-addc-d979008085b8 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:12.739946 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6b176-cbdf-4b2e-addc-d979008085b8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:12.740708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6b176-cbdf-4b2e-addc-d979008085b8 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:12.757553 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 371/747] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Oct 21 21:09:12 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:09:13.158140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96102dc6-f31e-4a5e-95af-477b0150bb20 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:13.158673 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96102dc6-f31e-4a5e-95af-477b0150bb20 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:13.158950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96102dc6-f31e-4a5e-95af-477b0150bb20 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:13.160690 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 377/748] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:13.165960 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b254a-f330-4a44-890a-ec40832e57ae None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:13.166351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b254a-f330-4a44-890a-ec40832e57ae None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:13.166721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b254a-f330-4a44-890a-ec40832e57ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:13.185370 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-0e9b254a-f330-4a44-890a-ec40832e57ae None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:13.186546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0e9b254a-f330-4a44-890a-ec40832e57ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:13.263369 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 372/749] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:13 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:13.536254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96102dc6-f31e-4a5e-95af-477b0150bb20 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:13.612670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:13.648244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96102dc6-f31e-4a5e-95af-477b0150bb20 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:13.683034 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756b9cb-32ad-479d-a32a-c56a99d055f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:13.683716 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756b9cb-32ad-479d-a32a-c56a99d055f5 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:13.683989 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756b9cb-32ad-479d-a32a-c56a99d055f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:13.684258 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756b9cb-32ad-479d-a32a-c56a99d055f5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:13.684848 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756b9cb-32ad-479d-a32a-c56a99d055f5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:13.686243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756b9cb-32ad-479d-a32a-c56a99d055f5 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:13.696679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-7756b9cb-32ad-479d-a32a-c56a99d055f5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 21:09:13.696679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:13.696679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:09:13.696679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:09:13.696679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:09:13.696679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:09:13.696679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:09:13.696679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:13.696679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 21:09:13.696679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:13.696679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:13.696679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:13.696679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:13.696679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:13.696679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:13.696679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:13.696679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:13.697702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:13.697702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:13.697702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:13.697702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:13.697702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:13.697702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:13.697702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:09:13.697702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:09:13.697702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:09:13.697702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:09:13.697702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:13.697702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:13.697702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:13.697702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:13.697702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:13.697702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:13.698540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:13.698540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:13.698540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:13.698540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:13.698540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:13.698540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:13.698540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:13.698540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:13.698540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:13.698540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:13.698540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:09:13.698540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:13.698540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 21:09:13.698540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:13.698540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 378/750] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:09:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:13.704100 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b254a-f330-4a44-890a-ec40832e57ae None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:13.732255 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bf663-d156-4a48-9718-2ee0fa1bac74 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:13.732881 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bf663-d156-4a48-9718-2ee0fa1bac74 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:13.733363 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bf663-d156-4a48-9718-2ee0fa1bac74 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:13.735155 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 379/751] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:13.740880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f29790-4673-4d17-af4d-decc80c044c6 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:13.742038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f29790-4673-4d17-af4d-decc80c044c6 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:13.742532 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f29790-4673-4d17-af4d-decc80c044c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:13.760858 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-b1f29790-4673-4d17-af4d-decc80c044c6 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:13.762346 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-b1f29790-4673-4d17-af4d-decc80c044c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:13.777528 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:13.818154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b254a-f330-4a44-890a-ec40832e57ae None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:13.841320 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 380/752] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:13 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 21:09:13.856437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec294f4-b145-4217-a280-c8aeead74efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:13.857047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec294f4-b145-4217-a280-c8aeead74efc None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:13.857330 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec294f4-b145-4217-a280-c8aeead74efc None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:13.857601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec294f4-b145-4217-a280-c8aeead74efc None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:13.858220 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec294f4-b145-4217-a280-c8aeead74efc None admin] RBAC: Adding query filter params (name=compute) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:13.858519 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec294f4-b145-4217-a280-c8aeead74efc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:13.859912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec294f4-b145-4217-a280-c8aeead74efc None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:13.865173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 373/753] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Oct 21 21:09:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:13.870792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f29790-4673-4d17-af4d-decc80c044c6 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:13.934256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:13.973038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f29790-4673-4d17-af4d-decc80c044c6 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:14.011727 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f8a4b-1b7d-4edc-9fe2-f241768c6304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:14.012913 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f8a4b-1b7d-4edc-9fe2-f241768c6304 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:14.013376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f8a4b-1b7d-4edc-9fe2-f241768c6304 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:14.013816 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f8a4b-1b7d-4edc-9fe2-f241768c6304 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:14.014621 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f8a4b-1b7d-4edc-9fe2-f241768c6304 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:14.015090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f8a4b-1b7d-4edc-9fe2-f241768c6304 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:14.017222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f8a4b-1b7d-4edc-9fe2-f241768c6304 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:14.022655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 381/754] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Oct 21 21:09:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:14.028221 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec294f4-b145-4217-a280-c8aeead74efc None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:14.118574 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f8a4b-1b7d-4edc-9fe2-f241768c6304 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:14.130915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec294f4-b145-4217-a280-c8aeead74efc None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:14.158625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:14.167994 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c810b65-7ef7-40c4-8c06-69a28b52d26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:14.168601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c810b65-7ef7-40c4-8c06-69a28b52d26b None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:14.168879 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c810b65-7ef7-40c4-8c06-69a28b52d26b None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:14.169149 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c810b65-7ef7-40c4-8c06-69a28b52d26b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:14.169843 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c810b65-7ef7-40c4-8c06-69a28b52d26b None admin] RBAC: Adding query filter params (interface=public, service_id=3be6a5afe6c7469d8ce198a61de273f7, region_id=RegionOne) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:14.170149 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c810b65-7ef7-40c4-8c06-69a28b52d26b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:14.171517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c810b65-7ef7-40c4-8c06-69a28b52d26b None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:14.176872 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 374/755] 10.222.0.13 () {56 vars in 1251 bytes} [Fri Oct 21 21:09:14 2022] GET /identity/v3/endpoints?service_id=3be6a5afe6c7469d8ce198a61de273f7&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:14.182272 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c810b65-7ef7-40c4-8c06-69a28b52d26b None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:14.259439 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f8a4b-1b7d-4edc-9fe2-f241768c6304 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:14.290273 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c810b65-7ef7-40c4-8c06-69a28b52d26b None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:14.300759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:14.312724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc298e30-3df6-473a-8c8b-466262fc8674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:14.313757 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc298e30-3df6-473a-8c8b-466262fc8674 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:14.314319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc298e30-3df6-473a-8c8b-466262fc8674 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:14.314762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc298e30-3df6-473a-8c8b-466262fc8674 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:14.315756 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc298e30-3df6-473a-8c8b-466262fc8674 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:14.318019 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc298e30-3df6-473a-8c8b-466262fc8674 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:14.328725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-bc298e30-3df6-473a-8c8b-466262fc8674 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 21:09:14.328725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:14.328725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:09:14.328725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:09:14.328725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:09:14.328725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:09:14.328725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:09:14.328725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:14.328725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 21:09:14.328725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:14.328725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:14.328725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:14.328725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:14.328725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:14.328725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:14.328725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:14.328725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:14.330470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:14.330470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:14.330470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:14.330470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:14.330470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:14.330470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:14.330470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:09:14.330470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:09:14.330470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:09:14.330470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:09:14.330470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:14.330470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:14.330470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:14.330470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:14.330470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:14.330470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:14.331525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:14.331525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:14.331525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:14.331525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:14.331525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:14.331525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:14.331525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:14.331525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:14.331525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:14.331525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:14.331525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:09:14.331525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:14.331525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 21:09:14.331525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:14.338120 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 382/756] 10.222.0.13 () {56 vars in 1126 bytes} [Fri Oct 21 21:09:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:09:14.339395 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc298e30-3df6-473a-8c8b-466262fc8674 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:14.339718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2e42d-dd13-4333-8351-0ed4608386d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:14.340333 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2e42d-dd13-4333-8351-0ed4608386d4 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:14.340615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2e42d-dd13-4333-8351-0ed4608386d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:14.340885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2e42d-dd13-4333-8351-0ed4608386d4 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:14.341485 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2e42d-dd13-4333-8351-0ed4608386d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:14.342846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2e42d-dd13-4333-8351-0ed4608386d4 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:14.355717 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 375/757] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Oct 21 21:09:14 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:09:14.376153 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf455f-a20c-4e3f-9079-bb2b687a8e99 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:14.376330 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf455f-a20c-4e3f-9079-bb2b687a8e99 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:14.376605 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf455f-a20c-4e3f-9079-bb2b687a8e99 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:14.378231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 376/758] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:09:14.384013 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4b8e0-4f5a-4144-a638-999f18b37621 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:14.384303 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4b8e0-4f5a-4144-a638-999f18b37621 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:14.384563 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4b8e0-4f5a-4144-a638-999f18b37621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:14.405070 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-abe4b8e0-4f5a-4144-a638-999f18b37621 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:14.406285 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-abe4b8e0-4f5a-4144-a638-999f18b37621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:14.467403 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:14.476905 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc298e30-3df6-473a-8c8b-466262fc8674 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:14.492471 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 377/759] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:14 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 21:09:14.522057 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdefa4-aabc-4de8-a663-ac5fe7a99f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:14.523108 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdefa4-aabc-4de8-a663-ac5fe7a99f1b None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:14.523639 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdefa4-aabc-4de8-a663-ac5fe7a99f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:14.524143 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdefa4-aabc-4de8-a663-ac5fe7a99f1b None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:14.525221 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bdefa4-aabc-4de8-a663-ac5fe7a99f1b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:14.525944 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bdefa4-aabc-4de8-a663-ac5fe7a99f1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:14.528260 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bdefa4-aabc-4de8-a663-ac5fe7a99f1b None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:14.534511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 383/760] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Oct 21 21:09:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:14.540509 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4b8e0-4f5a-4144-a638-999f18b37621 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:14.648383 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4b8e0-4f5a-4144-a638-999f18b37621 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:14.684253 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e882458-e4c7-446b-832d-d77dbc3854ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:14.684869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e882458-e4c7-446b-832d-d77dbc3854ca None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:14.685147 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e882458-e4c7-446b-832d-d77dbc3854ca None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:14.685419 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e882458-e4c7-446b-832d-d77dbc3854ca None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:14.686042 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e882458-e4c7-446b-832d-d77dbc3854ca None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:14.686343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e882458-e4c7-446b-832d-d77dbc3854ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:14.688788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e882458-e4c7-446b-832d-d77dbc3854ca None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:14.694452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 378/761] 10.222.0.13 () {56 vars in 1122 bytes} [Fri Oct 21 21:09:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:14.699986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdefa4-aabc-4de8-a663-ac5fe7a99f1b None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:14.762338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:14.794590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e882458-e4c7-446b-832d-d77dbc3854ca None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:14.809708 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdefa4-aabc-4de8-a663-ac5fe7a99f1b None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:14.844433 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49829c85-aff9-4ebf-97b9-bca58ebd67d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:14.845821 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49829c85-aff9-4ebf-97b9-bca58ebd67d7 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:14.846246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49829c85-aff9-4ebf-97b9-bca58ebd67d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:14.846662 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49829c85-aff9-4ebf-97b9-bca58ebd67d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:14.847484 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49829c85-aff9-4ebf-97b9-bca58ebd67d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:14.849030 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49829c85-aff9-4ebf-97b9-bca58ebd67d7 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:14.868227 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:14.869450 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 384/762] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:09:14 2022] POST /identity/v3/endpoints => generated 325 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:09:14.913845 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e882458-e4c7-446b-832d-d77dbc3854ca None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:14.963339 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a69d19-8fbc-4e0a-bd57-b175e5ea00cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:14.964001 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a69d19-8fbc-4e0a-bd57-b175e5ea00cd None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:14.964554 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a69d19-8fbc-4e0a-bd57-b175e5ea00cd None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:14.964824 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a69d19-8fbc-4e0a-bd57-b175e5ea00cd None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:14.965483 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a69d19-8fbc-4e0a-bd57-b175e5ea00cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:14.966968 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a69d19-8fbc-4e0a-bd57-b175e5ea00cd None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:14.970207 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: WARNING keystone.server.flask.common [None req-03a69d19-8fbc-4e0a-bd57-b175e5ea00cd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 21 21:09:14.983631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.sql.core [None req-03a69d19-8fbc-4e0a-bd57-b175e5ea00cd None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 21 21:09:14.983631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 21 21:09:14.983631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [parameters: {'user_id': '71b3a4c59fef45238e10b05286aa7331', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 21 21:09:14.983631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81306) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 21 21:09:15.001240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-03a69d19-8fbc-4e0a-bd57-b175e5ea00cd None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 21 21:09:15.001240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:15.001240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 21:09:15.001240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 21:09:15.001240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 21:09:15.001240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 21:09:15.001240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 21:09:15.001240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 21:09:15.001240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 21:09:15.001240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application conn.query(q) Oct 21 21:09:15.001240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 21:09:15.001240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 21:09:15.001240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 21:09:15.001240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application result.read() Oct 21 21:09:15.001240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 21:09:15.002181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application next(self.gen) Oct 21 21:09:15.003095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 21 21:09:15.003095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application yield resource Oct 21 21:09:15.003095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 21:09:15.003095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application next(self.gen) Oct 21 21:09:15.003095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 21 21:09:15.003095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self.session.rollback() Oct 21 21:09:15.003095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 21 21:09:15.003095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self.force_reraise() Oct 21 21:09:15.003095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 21 21:09:15.003095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise self.value Oct 21 21:09:15.003095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 21 21:09:15.003095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 21 21:09:15.003095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 21 21:09:15.003095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application session.commit() Oct 21 21:09:15.003095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 21 21:09:15.003095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 21 21:09:15.003095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 21 21:09:15.003970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._prepare_impl() Oct 21 21:09:15.003970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 21 21:09:15.003970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self.session.flush() Oct 21 21:09:15.003970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 21 21:09:15.003970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._flush(objects) Oct 21 21:09:15.003970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 21 21:09:15.003970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 21 21:09:15.003970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 21 21:09:15.003970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application compat.raise_( Oct 21 21:09:15.003970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 21:09:15.003970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception Oct 21 21:09:15.003970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 21 21:09:15.003970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application flush_context.execute() Oct 21 21:09:15.003970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 21 21:09:15.003970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rec.execute(self) Oct 21 21:09:15.003970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 21 21:09:15.003970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 21 21:09:15.004855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 21 21:09:15.004855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 21 21:09:15.004855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 21 21:09:15.004855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 21 21:09:15.004855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 21 21:09:15.004855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 21 21:09:15.004855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 21 21:09:15.004855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 21 21:09:15.004855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 21 21:09:15.004855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 21 21:09:15.004855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 21 21:09:15.004855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 21 21:09:15.004855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 21 21:09:15.004855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 21 21:09:15.004855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 21:09:15.004855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception Oct 21 21:09:15.005675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 21:09:15.005675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 21:09:15.005675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 21:09:15.005675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 21:09:15.005675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 21:09:15.005675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 21:09:15.005675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 21:09:15.005675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application conn.query(q) Oct 21 21:09:15.005675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 21:09:15.005675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 21:09:15.005675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 21:09:15.005675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application result.read() Oct 21 21:09:15.005675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 21:09:15.005675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 21:09:15.005675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 21:09:15.005675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 21:09:15.006476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 21:09:15.006476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 21:09:15.006476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 21:09:15.006476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 21:09:15.006476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 21 21:09:15.006476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 21 21:09:15.006476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [parameters: {'user_id': '71b3a4c59fef45238e10b05286aa7331', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 21 21:09:15.006476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 21 21:09:15.006476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:15.006476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:15.006476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:15.006476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:15.006476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:15.006476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:15.006476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:15.007267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:15.007267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:15.007267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:15.007267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:15.007267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:15.007267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:15.007267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:15.007267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 21 21:09:15.007267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 21 21:09:15.007267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:15.007267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:15.007267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:09:15.007267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:09:15.007267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:09:15.007267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:09:15.007267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 21 21:09:15.007267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 21 21:09:15.010566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 21 21:09:15.010566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 21 21:09:15.010566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 21 21:09:15.010566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 21 21:09:15.010566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 21 21:09:15.010566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:15.010566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 379/763] 10.222.0.13 () {60 vars in 1131 bytes} [Fri Oct 21 21:09:14 2022] POST /identity/v3/users => generated 160 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:09:15.012875 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49829c85-aff9-4ebf-97b9-bca58ebd67d7 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:15.058185 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e267b39-2ade-4a66-9b8b-1d7813083cce None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:15.058482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e267b39-2ade-4a66-9b8b-1d7813083cce None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:15.058767 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e267b39-2ade-4a66-9b8b-1d7813083cce None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:15.067175 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 380/764] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:09:15.068444 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f286a4d-32c1-4e2c-9c52-3c13830accef None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:15.068725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f286a4d-32c1-4e2c-9c52-3c13830accef None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:15.068999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f286a4d-32c1-4e2c-9c52-3c13830accef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:15.090280 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-8f286a4d-32c1-4e2c-9c52-3c13830accef None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:15.091996 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8f286a4d-32c1-4e2c-9c52-3c13830accef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:15.150424 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49829c85-aff9-4ebf-97b9-bca58ebd67d7 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:15.169195 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:15.175807 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 381/765] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:15 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:15.195799 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6088c2f-78f6-4dc1-b5d9-080b21cb5639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:15.196434 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6088c2f-78f6-4dc1-b5d9-080b21cb5639 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:15.196710 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6088c2f-78f6-4dc1-b5d9-080b21cb5639 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:15.196980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6088c2f-78f6-4dc1-b5d9-080b21cb5639 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:15.205712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6088c2f-78f6-4dc1-b5d9-080b21cb5639 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:15.207141 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6088c2f-78f6-4dc1-b5d9-080b21cb5639 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:15.219001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-e6088c2f-78f6-4dc1-b5d9-080b21cb5639 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 21 21:09:15.219001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:15.219001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:15.219001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:15.219001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:15.219001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:15.219001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:15.219001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:15.219001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:15.219001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:15.219001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:15.219001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:15.219001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:09:15.219001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:09:15.219001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:09:15.219001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:15.220233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:15.221600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:15.221600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:15.221600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:09:15.221600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:09:15.221600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:09:15.221600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:09:15.221600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:09:15.221600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:09:15.221600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 21 21:09:15.221600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:15.221600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 385/766] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:09:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:15.226200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f286a4d-32c1-4e2c-9c52-3c13830accef None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:15.266882 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:15.331823 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f286a4d-32c1-4e2c-9c52-3c13830accef None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:15.366454 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a395c-f63a-4324-b41f-0fdeb1d68012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:15.367069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a395c-f63a-4324-b41f-0fdeb1d68012 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:15.367349 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a395c-f63a-4324-b41f-0fdeb1d68012 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:15.367654 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a395c-f63a-4324-b41f-0fdeb1d68012 None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:15.368345 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a395c-f63a-4324-b41f-0fdeb1d68012 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:15.368646 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a395c-f63a-4324-b41f-0fdeb1d68012 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:15.370276 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a395c-f63a-4324-b41f-0fdeb1d68012 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:15.413827 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 382/767] 10.222.0.13 () {56 vars in 1101 bytes} [Fri Oct 21 21:09:15 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:15.483706 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6088c2f-78f6-4dc1-b5d9-080b21cb5639 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:15.597409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6088c2f-78f6-4dc1-b5d9-080b21cb5639 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:15.633895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f1712-1ea7-472c-a6f4-1447fd9272c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:15.634528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f1712-1ea7-472c-a6f4-1447fd9272c8 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:15.634801 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f1712-1ea7-472c-a6f4-1447fd9272c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:15.635070 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f1712-1ea7-472c-a6f4-1447fd9272c8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:15.635682 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f1712-1ea7-472c-a6f4-1447fd9272c8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:15.637053 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f1712-1ea7-472c-a6f4-1447fd9272c8 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:15.646452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-1f5f1712-1ea7-472c-a6f4-1447fd9272c8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 21:09:15.646452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:15.646452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:09:15.646452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:09:15.646452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:09:15.646452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:09:15.646452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:09:15.646452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:15.646452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 21:09:15.646452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:15.646452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:15.646452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:15.646452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:15.646452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:15.646452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:15.646452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:15.646452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:15.647482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:15.647482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:15.647482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:15.647482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:15.647482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:15.647482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:15.647482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:09:15.647482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:09:15.647482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:09:15.647482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:09:15.647482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:15.647482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:15.647482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:15.647482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:15.647482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:15.647482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:15.648353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:15.648353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:15.648353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:15.648353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:15.648353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:15.648353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:15.648353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:15.648353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:15.648353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:15.648353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:15.648353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:09:15.648353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:15.648353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 21:09:15.648353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:15.648353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 386/768] 10.222.0.13 () {56 vars in 1105 bytes} [Fri Oct 21 21:09:15 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:15.655335 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a395c-f63a-4324-b41f-0fdeb1d68012 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:15.764484 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:15.771557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a395c-f63a-4324-b41f-0fdeb1d68012 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:15.805364 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9ac47-b132-42e6-8bf6-2aafd7663572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:15.806031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9ac47-b132-42e6-8bf6-2aafd7663572 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:15.806230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9ac47-b132-42e6-8bf6-2aafd7663572 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:15.806485 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9ac47-b132-42e6-8bf6-2aafd7663572 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:15.807103 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9ac47-b132-42e6-8bf6-2aafd7663572 None admin] RBAC: Adding query filter params (name=image) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:15.807399 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9ac47-b132-42e6-8bf6-2aafd7663572 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:15.808776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9ac47-b132-42e6-8bf6-2aafd7663572 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:15.813878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 383/769] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:15.819242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f1712-1ea7-472c-a6f4-1447fd9272c8 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:15.861460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:15.920965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f1712-1ea7-472c-a6f4-1447fd9272c8 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:15.952681 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8a390-edf5-4ee7-8689-2fcd9666ae43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:15.953281 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8a390-edf5-4ee7-8689-2fcd9666ae43 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:15.953557 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8a390-edf5-4ee7-8689-2fcd9666ae43 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:15.954330 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8a390-edf5-4ee7-8689-2fcd9666ae43 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:15.954954 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8a390-edf5-4ee7-8689-2fcd9666ae43 None admin] RBAC: Adding query filter params (type=image) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:15.955255 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8a390-edf5-4ee7-8689-2fcd9666ae43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:15.956631 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8a390-edf5-4ee7-8689-2fcd9666ae43 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:15.961485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 387/770] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:15 2022] GET /identity/v3/services?type=image => generated 350 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:15.967068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9ac47-b132-42e6-8bf6-2aafd7663572 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:16.064798 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9ac47-b132-42e6-8bf6-2aafd7663572 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:16.099206 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c29ec0-9e5f-4a84-a217-fd9b073b64cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:16.099893 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c29ec0-9e5f-4a84-a217-fd9b073b64cc None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:16.100180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c29ec0-9e5f-4a84-a217-fd9b073b64cc None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:16.100450 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c29ec0-9e5f-4a84-a217-fd9b073b64cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:16.101141 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c29ec0-9e5f-4a84-a217-fd9b073b64cc None admin] RBAC: Adding query filter params (interface=public, service_id=8d563494425f494a81093292daae336f, region_id=RegionOne) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:16.101446 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c29ec0-9e5f-4a84-a217-fd9b073b64cc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:16.102816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c29ec0-9e5f-4a84-a217-fd9b073b64cc None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:16.112574 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 384/771] 10.222.0.13 () {56 vars in 1251 bytes} [Fri Oct 21 21:09:15 2022] GET /identity/v3/endpoints?service_id=8d563494425f494a81093292daae336f&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:09:16.114113 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8a390-edf5-4ee7-8689-2fcd9666ae43 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:16.214240 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8a390-edf5-4ee7-8689-2fcd9666ae43 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:16.216677 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:16.248989 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c3b1d-b280-43b3-94f8-43f2e4c15a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:16.249599 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c3b1d-b280-43b3-94f8-43f2e4c15a4f None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:16.249881 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c3b1d-b280-43b3-94f8-43f2e4c15a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:16.250153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c3b1d-b280-43b3-94f8-43f2e4c15a4f None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:16.251275 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c3b1d-b280-43b3-94f8-43f2e4c15a4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:16.252658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c3b1d-b280-43b3-94f8-43f2e4c15a4f None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:16.267676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 388/772] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Oct 21 21:09:16 2022] GET /identity/v3/services => generated 1957 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:09:16.599198 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb4c6f-ac4f-4ca0-98a3-2bd5dbe51df4 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:16.599591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb4c6f-ac4f-4ca0-98a3-2bd5dbe51df4 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:16.599796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb4c6f-ac4f-4ca0-98a3-2bd5dbe51df4 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:16.601278 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 385/773] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:16.606321 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865104d-9efb-44e2-a0df-47c4410bf800 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:16.606775 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865104d-9efb-44e2-a0df-47c4410bf800 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:16.606861 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865104d-9efb-44e2-a0df-47c4410bf800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:16.625340 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-6865104d-9efb-44e2-a0df-47c4410bf800 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:16.626586 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-6865104d-9efb-44e2-a0df-47c4410bf800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:16.692792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 389/774] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:16 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:16.960525 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb4c6f-ac4f-4ca0-98a3-2bd5dbe51df4 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:16.985990 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:17.068838 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb4c6f-ac4f-4ca0-98a3-2bd5dbe51df4 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:17.102086 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b16c6-d019-4621-af6d-b7ab3e37ecda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:17.102722 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b16c6-d019-4621-af6d-b7ab3e37ecda None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:17.102995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b16c6-d019-4621-af6d-b7ab3e37ecda None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:17.103267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b16c6-d019-4621-af6d-b7ab3e37ecda None admin] PATH_INFO: `/v3/services/compute` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:17.103860 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b16c6-d019-4621-af6d-b7ab3e37ecda None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:17.105240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b16c6-d019-4621-af6d-b7ab3e37ecda None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:17.114026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-e48b16c6-d019-4621-af6d-b7ab3e37ecda None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 21:09:17.114026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:17.114026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:09:17.114026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:09:17.114026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:09:17.114026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:09:17.114026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:09:17.114026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:17.114026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 21:09:17.114026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:17.114026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:17.114026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:17.114026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:17.114026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:17.114026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:17.114026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:17.114026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:17.115035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:17.115035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:17.115035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:17.115035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:17.115035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:17.115035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:17.115035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:09:17.115035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:09:17.115035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:09:17.115035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:09:17.115035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:17.115035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:17.115035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:17.115035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:17.115035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:17.115035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:17.115896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:17.115896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:17.115896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:17.115896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:17.115896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:17.115896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:17.115896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:17.115896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:17.115896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:17.115896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:17.115896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:09:17.115896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:17.115896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 21:09:17.115896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:17.117556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 386/775] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:09:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:17.121013 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865104d-9efb-44e2-a0df-47c4410bf800 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:17.136866 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb931137-d353-4b64-919f-3207ee3361df None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:17.137148 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb931137-d353-4b64-919f-3207ee3361df None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:17.137418 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb931137-d353-4b64-919f-3207ee3361df None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:17.139001 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 387/776] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:09:17.144698 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752b795-a916-41b6-8770-b7fd43243a44 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:17.144974 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752b795-a916-41b6-8770-b7fd43243a44 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:17.145242 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752b795-a916-41b6-8770-b7fd43243a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:17.166226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-0752b795-a916-41b6-8770-b7fd43243a44 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:17.167432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0752b795-a916-41b6-8770-b7fd43243a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:17.226234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:17.230577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865104d-9efb-44e2-a0df-47c4410bf800 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:17.238801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 388/777] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:17 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 21:09:17.265887 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde37f1-6a4c-4acf-8ba9-9d15a08894c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:17.266982 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde37f1-6a4c-4acf-8ba9-9d15a08894c3 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:17.267256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde37f1-6a4c-4acf-8ba9-9d15a08894c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:17.267523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde37f1-6a4c-4acf-8ba9-9d15a08894c3 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:17.268167 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde37f1-6a4c-4acf-8ba9-9d15a08894c3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:17.268466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde37f1-6a4c-4acf-8ba9-9d15a08894c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:17.269803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde37f1-6a4c-4acf-8ba9-9d15a08894c3 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:17.279184 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 390/778] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Oct 21 21:09:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:17.280416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752b795-a916-41b6-8770-b7fd43243a44 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:17.300411 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:17.381536 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752b795-a916-41b6-8770-b7fd43243a44 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:17.414809 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a3e55-bb3a-4971-ae81-248d289012a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:17.415419 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a3e55-bb3a-4971-ae81-248d289012a2 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:17.415724 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a3e55-bb3a-4971-ae81-248d289012a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:17.415995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a3e55-bb3a-4971-ae81-248d289012a2 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:17.416612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a3e55-bb3a-4971-ae81-248d289012a2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:17.416915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a3e55-bb3a-4971-ae81-248d289012a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:17.418277 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a3e55-bb3a-4971-ae81-248d289012a2 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:17.428265 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 389/779] 10.222.0.13 () {56 vars in 1112 bytes} [Fri Oct 21 21:09:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:17.429874 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde37f1-6a4c-4acf-8ba9-9d15a08894c3 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:17.502961 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a3e55-bb3a-4971-ae81-248d289012a2 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:17.532845 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde37f1-6a4c-4acf-8ba9-9d15a08894c3 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:17.569101 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87cb23-2253-4596-ae5c-2eb35fb62f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:17.569703 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87cb23-2253-4596-ae5c-2eb35fb62f17 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:17.569984 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87cb23-2253-4596-ae5c-2eb35fb62f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:17.570257 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87cb23-2253-4596-ae5c-2eb35fb62f17 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:17.570918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87cb23-2253-4596-ae5c-2eb35fb62f17 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:17.572304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87cb23-2253-4596-ae5c-2eb35fb62f17 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:17.588919 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:17.594100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 391/780] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:09:17 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:09:17.610167 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a3e55-bb3a-4971-ae81-248d289012a2 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:17.656693 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3837708-f372-4c4a-8715-06f1df76f728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:17.657375 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3837708-f372-4c4a-8715-06f1df76f728 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:17.657651 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3837708-f372-4c4a-8715-06f1df76f728 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:17.657918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3837708-f372-4c4a-8715-06f1df76f728 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:17.659068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3837708-f372-4c4a-8715-06f1df76f728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:17.660546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3837708-f372-4c4a-8715-06f1df76f728 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:17.667319 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 390/781] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:09:17.668161 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87cb23-2253-4596-ae5c-2eb35fb62f17 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:17.689738 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:17.787807 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87cb23-2253-4596-ae5c-2eb35fb62f17 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:17.823105 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde2bac-b628-45a0-88a8-19bb519b1bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:17.823764 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde2bac-b628-45a0-88a8-19bb519b1bd9 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:17.824044 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde2bac-b628-45a0-88a8-19bb519b1bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:17.824321 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde2bac-b628-45a0-88a8-19bb519b1bd9 None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:17.824982 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde2bac-b628-45a0-88a8-19bb519b1bd9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:17.826390 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde2bac-b628-45a0-88a8-19bb519b1bd9 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:17.855765 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 392/782] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Oct 21 21:09:17 2022] POST /identity/v3/projects => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:09:18.437048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9e3b8-1c0c-4003-83a4-cddf7317059a None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:18.437572 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9e3b8-1c0c-4003-83a4-cddf7317059a None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:18.437572 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9e3b8-1c0c-4003-83a4-cddf7317059a None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:18.439541 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 391/783] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:18.444772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b56622-3c5e-4f39-be5d-e2e3e3ab41ec None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:18.445156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b56622-3c5e-4f39-be5d-e2e3e3ab41ec None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:18.445428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b56622-3c5e-4f39-be5d-e2e3e3ab41ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:18.463829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-33b56622-3c5e-4f39-be5d-e2e3e3ab41ec None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:18.465009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-33b56622-3c5e-4f39-be5d-e2e3e3ab41ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:18.528277 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:18.539852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 393/784] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:18.801179 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9e3b8-1c0c-4003-83a4-cddf7317059a None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:18.901716 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9e3b8-1c0c-4003-83a4-cddf7317059a None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:18.935043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1db89-0b51-4afd-80b4-efb04372cb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:18.935731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1db89-0b51-4afd-80b4-efb04372cb52 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:18.936007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1db89-0b51-4afd-80b4-efb04372cb52 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:18.936272 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1db89-0b51-4afd-80b4-efb04372cb52 None admin] PATH_INFO: `/v3/services/image` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:18.936831 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1db89-0b51-4afd-80b4-efb04372cb52 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:18.938225 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1db89-0b51-4afd-80b4-efb04372cb52 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:18.946839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-aac1db89-0b51-4afd-80b4-efb04372cb52 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 21:09:18.946839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:18.946839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:09:18.946839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:09:18.946839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:09:18.946839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:09:18.946839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:09:18.946839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:18.946839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 21:09:18.946839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:18.946839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:18.946839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:18.946839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:18.946839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:18.946839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:18.946839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:18.946839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:18.948072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:18.948072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:18.948072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:18.948072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:18.948072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:18.948072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:18.948072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:09:18.948072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:09:18.948072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:09:18.948072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:09:18.948072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:18.948072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:18.948072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:18.948072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:18.948072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:18.948072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:18.948913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:18.948913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:18.948913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:18.948913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:18.948913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:18.948913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:18.948913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:18.948913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:18.948913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:18.948913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:18.948913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:09:18.948913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:09:18.948913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 21:09:18.948913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:18.952972 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 392/785] 10.222.0.13 () {56 vars in 1105 bytes} [Fri Oct 21 21:09:18 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:18.954695 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b56622-3c5e-4f39-be5d-e2e3e3ab41ec None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:19.054037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b56622-3c5e-4f39-be5d-e2e3e3ab41ec None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:19.087336 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bce9b6-26c7-4348-b645-d47a443a0ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:19.087972 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bce9b6-26c7-4348-b645-d47a443a0ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:19.088422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bce9b6-26c7-4348-b645-d47a443a0ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:19.088701 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bce9b6-26c7-4348-b645-d47a443a0ad5 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:19.089325 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bce9b6-26c7-4348-b645-d47a443a0ad5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:19.089630 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bce9b6-26c7-4348-b645-d47a443a0ad5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:19.090983 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bce9b6-26c7-4348-b645-d47a443a0ad5 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:19.100527 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 394/786] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:19.101766 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1db89-0b51-4afd-80b4-efb04372cb52 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:19.114775 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:19.202356 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1db89-0b51-4afd-80b4-efb04372cb52 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:19.234585 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966a5a0-0c93-484f-904d-14149ffcf83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:19.235211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966a5a0-0c93-484f-904d-14149ffcf83e None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:19.235485 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966a5a0-0c93-484f-904d-14149ffcf83e None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:19.235778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966a5a0-0c93-484f-904d-14149ffcf83e None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:19.236393 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966a5a0-0c93-484f-904d-14149ffcf83e None admin] RBAC: Adding query filter params (type=image) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:19.236689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966a5a0-0c93-484f-904d-14149ffcf83e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:19.238041 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966a5a0-0c93-484f-904d-14149ffcf83e None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:19.242967 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 393/787] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:19.248442 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bce9b6-26c7-4348-b645-d47a443a0ad5 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:19.309086 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:19.347351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bce9b6-26c7-4348-b645-d47a443a0ad5 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:19.380269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f2839-375c-4fe1-a80a-5c07de68efd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:19.380861 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f2839-375c-4fe1-a80a-5c07de68efd2 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:19.381134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f2839-375c-4fe1-a80a-5c07de68efd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:19.381401 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f2839-375c-4fe1-a80a-5c07de68efd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:19.382066 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f2839-375c-4fe1-a80a-5c07de68efd2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:19.383400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f2839-375c-4fe1-a80a-5c07de68efd2 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:19.403925 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 395/788] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:09:19 2022] POST /identity/v3/endpoints => generated 327 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:09:19.832814 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2891401-f754-43d1-9c09-bc2ad6fa4573 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:19.833329 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2891401-f754-43d1-9c09-bc2ad6fa4573 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:19.833431 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2891401-f754-43d1-9c09-bc2ad6fa4573 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:19.834883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 394/789] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:19.839800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7d103-f477-4a2d-98e5-c3414b3a050f None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:19.840096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7d103-f477-4a2d-98e5-c3414b3a050f None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:19.840376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7d103-f477-4a2d-98e5-c3414b3a050f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:19.859102 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-5ca7d103-f477-4a2d-98e5-c3414b3a050f None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:19.860887 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-5ca7d103-f477-4a2d-98e5-c3414b3a050f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:19.937520 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 396/790] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:20.031087 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342f27f-547b-475a-be85-17e532702a6e None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:20.031398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342f27f-547b-475a-be85-17e532702a6e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:20.031593 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342f27f-547b-475a-be85-17e532702a6e None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:20.033574 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 395/791] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:20.038877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb96237-a67f-4dd1-85da-35c68c84fe19 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:20.039350 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb96237-a67f-4dd1-85da-35c68c84fe19 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:20.039429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb96237-a67f-4dd1-85da-35c68c84fe19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:20.057923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-deb96237-a67f-4dd1-85da-35c68c84fe19 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:20.059088 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-deb96237-a67f-4dd1-85da-35c68c84fe19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:20.101997 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:20.128792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 397/792] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 21:09:20.202631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342f27f-547b-475a-be85-17e532702a6e None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:20.277378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:20.304056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342f27f-547b-475a-be85-17e532702a6e None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:20.339345 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ea940-8204-4324-a572-ba314756c75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:20.340001 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea940-8204-4324-a572-ba314756c75c None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:20.340278 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea940-8204-4324-a572-ba314756c75c None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:20.340546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea940-8204-4324-a572-ba314756c75c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:20.350835 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ea940-8204-4324-a572-ba314756c75c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:09:20.355534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-e22ea940-8204-4324-a572-ba314756c75c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 21 21:09:20.355534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:20.355534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:20.355534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:20.355534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:20.355534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:20.355534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:20.355534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:20.355534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:20.355534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:20.355534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:20.355534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:20.355534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:09:20.355534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:09:20.355534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:09:20.355534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:09:20.356512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:09:20.356512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:09:20.356512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:09:20.356512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:09:20.356512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:20.356512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:20.356512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:20.356512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:20.356512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:20.356512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:20.356512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:20.356512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:20.356512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:20.356512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:20.356512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:20.356512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:20.357341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:20.357341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:20.357341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:20.357341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:20.357341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:09:20.357341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:09:20.357341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:09:20.357341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:09:20.357341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:09:20.357341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:09:20.357341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 21 21:09:20.357341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:20.360387 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 396/793] 10.222.0.13 () {56 vars in 1120 bytes} [Fri Oct 21 21:09:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:20.363050 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb96237-a67f-4dd1-85da-35c68c84fe19 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:20.394192 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ea940-8204-4324-a572-ba314756c75c None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:20.463870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb96237-a67f-4dd1-85da-35c68c84fe19 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:20.497556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ea940-8204-4324-a572-ba314756c75c None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:20.502165 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e316b-c956-426d-9f01-35cebb68dbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:20.502767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e316b-c956-426d-9f01-35cebb68dbc9 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:20.503042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e316b-c956-426d-9f01-35cebb68dbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:20.503313 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e316b-c956-426d-9f01-35cebb68dbc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:20.503983 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e316b-c956-426d-9f01-35cebb68dbc9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:20.504339 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e316b-c956-426d-9f01-35cebb68dbc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:20.505820 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e316b-c956-426d-9f01-35cebb68dbc9 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:20.515597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 398/794] 10.222.0.13 () {56 vars in 1115 bytes} [Fri Oct 21 21:09:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:20.520075 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e316b-c956-426d-9f01-35cebb68dbc9 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:20.534369 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ac547-869a-4517-9bb6-0c1fd60cab14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:20.535016 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ac547-869a-4517-9bb6-0c1fd60cab14 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:20.535288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ac547-869a-4517-9bb6-0c1fd60cab14 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:20.535560 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ac547-869a-4517-9bb6-0c1fd60cab14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:20.536693 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ac547-869a-4517-9bb6-0c1fd60cab14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:20.538142 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ac547-869a-4517-9bb6-0c1fd60cab14 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:20.539771 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 397/795] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:09:20.545197 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ac547-869a-4517-9bb6-0c1fd60cab14 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:20.548338 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:20.567352 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:20.631146 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e316b-c956-426d-9f01-35cebb68dbc9 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:20.653223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ac547-869a-4517-9bb6-0c1fd60cab14 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:20.667243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3c8d7-8978-4275-8161-58d9af68e335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:20.667899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3c8d7-8978-4275-8161-58d9af68e335 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:20.668174 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3c8d7-8978-4275-8161-58d9af68e335 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:20.668439 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3c8d7-8978-4275-8161-58d9af68e335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:20.679182 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3c8d7-8978-4275-8161-58d9af68e335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:20.680624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3c8d7-8978-4275-8161-58d9af68e335 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:20.687513 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbe744-e919-47e6-a682-70b89cd72abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:20.688690 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbe744-e919-47e6-a682-70b89cd72abd None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:20.688977 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbe744-e919-47e6-a682-70b89cd72abd None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:20.689249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbe744-e919-47e6-a682-70b89cd72abd None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:20.689896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fbe744-e919-47e6-a682-70b89cd72abd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:20.691339 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fbe744-e919-47e6-a682-70b89cd72abd None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:20.695405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-4bf3c8d7-8978-4275-8161-58d9af68e335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:20.695405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:20.695405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:20.695405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:20.695405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:20.695405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:20.695405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:20.695405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:20.695405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:20.695405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:20.695405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:20.695405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:20.695405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:09:20.695405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:09:20.695405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:09:20.695405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:09:20.696377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:20.696377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:20.696377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:20.696377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:20.696377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:20.696377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:20.696377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:20.696377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:20.696377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:20.696377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:20.696377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:20.696377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:20.696377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:20.696377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:20.696377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:20.696377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:20.696377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:09:20.697779 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:09:20.697779 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:20.697779 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:20.697779 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 399/796] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:09:20.702101 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3c8d7-8978-4275-8161-58d9af68e335 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:20.715612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 398/797] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:09:20 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:09:20.804683 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:20.815420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3c8d7-8978-4275-8161-58d9af68e335 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:20.852004 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c1e1f-1654-4929-b1ef-bfec80babe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:20.852532 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c1e1f-1654-4929-b1ef-bfec80babe13 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:20.852817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c1e1f-1654-4929-b1ef-bfec80babe13 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:20.853090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c1e1f-1654-4929-b1ef-bfec80babe13 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:20.853736 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c1e1f-1654-4929-b1ef-bfec80babe13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:20.854039 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c1e1f-1654-4929-b1ef-bfec80babe13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:20.855398 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c1e1f-1654-4929-b1ef-bfec80babe13 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:20.873685 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 400/798] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:09:20.875076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbe744-e919-47e6-a682-70b89cd72abd None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:20.936266 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:20.981135 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbe744-e919-47e6-a682-70b89cd72abd None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:21.015184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f48983-bbab-4cdb-b16b-969bfe89c5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:21.015837 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f48983-bbab-4cdb-b16b-969bfe89c5ef None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:21.016648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f48983-bbab-4cdb-b16b-969bfe89c5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:21.016915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f48983-bbab-4cdb-b16b-969bfe89c5ef None admin] PATH_INFO: `/v3/users/nova` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:21.024312 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f48983-bbab-4cdb-b16b-969bfe89c5ef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:21.025740 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f48983-bbab-4cdb-b16b-969bfe89c5ef None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:21.037066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-a9f48983-bbab-4cdb-b16b-969bfe89c5ef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 21 21:09:21.037066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:21.037066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:21.037066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:21.037066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:21.037066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:21.037066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:21.037066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:21.037066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:21.037066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:21.037066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:21.037066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:21.037066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:09:21.037066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:09:21.037066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:09:21.037066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:21.037998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:21.039362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:21.039362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:21.039362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:09:21.039362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:09:21.039362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:09:21.039362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:09:21.039362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:09:21.039362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:09:21.039362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 21 21:09:21.039362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:21.039362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 399/799] 10.222.0.13 () {56 vars in 1093 bytes} [Fri Oct 21 21:09:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:21.044874 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c1e1f-1654-4929-b1ef-bfec80babe13 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:21.146195 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c1e1f-1654-4929-b1ef-bfec80babe13 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:21.180522 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30f5c0-22af-47b7-a354-876d8e23b089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:21.181136 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30f5c0-22af-47b7-a354-876d8e23b089 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:21.181414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30f5c0-22af-47b7-a354-876d8e23b089 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:21.181771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce30f5c0-22af-47b7-a354-876d8e23b089 None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:21.182503 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30f5c0-22af-47b7-a354-876d8e23b089 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:21.182857 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30f5c0-22af-47b7-a354-876d8e23b089 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:21.184379 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce30f5c0-22af-47b7-a354-876d8e23b089 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:21.204209 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 401/800] 10.222.0.13 () {56 vars in 1133 bytes} [Fri Oct 21 21:09:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:21.208275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f48983-bbab-4cdb-b16b-969bfe89c5ef None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:21.307214 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f48983-bbab-4cdb-b16b-969bfe89c5ef None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:21.324038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:21.341712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac8ead-852e-4a78-b874-e0384fcd43f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:21.342341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac8ead-852e-4a78-b874-e0384fcd43f2 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:21.342615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac8ead-852e-4a78-b874-e0384fcd43f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:21.342881 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac8ead-852e-4a78-b874-e0384fcd43f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:21.353668 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac8ead-852e-4a78-b874-e0384fcd43f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:21.355100 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac8ead-852e-4a78-b874-e0384fcd43f2 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:21.368491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-8fac8ead-852e-4a78-b874-e0384fcd43f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:21.368491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:21.368491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:21.368491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:21.368491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:21.368491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:21.368491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:21.368491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:21.368491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:21.368491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:21.368491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:21.368491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:21.368491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:09:21.368491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:09:21.368491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:09:21.368491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:09:21.369415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:21.369415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:21.369415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:21.369415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:21.369415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:21.369415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:21.369415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:21.369415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:21.369415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:21.369415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:21.369415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:21.369415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:21.369415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:21.369415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:21.369415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:21.369415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:21.369415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:09:21.370856 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:09:21.370856 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:21.370856 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:21.370856 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 400/801] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:09:21.375197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30f5c0-22af-47b7-a354-876d8e23b089 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:21.386776 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:21.475895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce30f5c0-22af-47b7-a354-876d8e23b089 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:21.520794 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a27717-3be6-4942-8bd1-484076226ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:21.521421 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a27717-3be6-4942-8bd1-484076226ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:21.521710 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a27717-3be6-4942-8bd1-484076226ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:21.521973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a27717-3be6-4942-8bd1-484076226ee3 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:21.522606 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a27717-3be6-4942-8bd1-484076226ee3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:21.522898 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a27717-3be6-4942-8bd1-484076226ee3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:21.524275 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a27717-3be6-4942-8bd1-484076226ee3 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:21.535606 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 402/802] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:21.541375 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac8ead-852e-4a78-b874-e0384fcd43f2 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:21.606131 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b25091-aa9a-42c1-b52c-2b76048d1b9d None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:21.606548 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b25091-aa9a-42c1-b52c-2b76048d1b9d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:21.606666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b25091-aa9a-42c1-b52c-2b76048d1b9d None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:21.608189 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 403/803] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:21.613397 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b5b20-cc94-4138-a8db-aa7107bc043a None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:21.613684 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b5b20-cc94-4138-a8db-aa7107bc043a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:21.613958 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b5b20-cc94-4138-a8db-aa7107bc043a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:21.631539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-652b5b20-cc94-4138-a8db-aa7107bc043a None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:21.632746 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-652b5b20-cc94-4138-a8db-aa7107bc043a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:21.642191 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac8ead-852e-4a78-b874-e0384fcd43f2 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:21.677388 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bea7fd-e7cf-4ba2-b011-be2f99e8f519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:21.678024 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bea7fd-e7cf-4ba2-b011-be2f99e8f519 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:21.678300 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bea7fd-e7cf-4ba2-b011-be2f99e8f519 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:21.678567 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bea7fd-e7cf-4ba2-b011-be2f99e8f519 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:21.685082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bea7fd-e7cf-4ba2-b011-be2f99e8f519 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:21.686515 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bea7fd-e7cf-4ba2-b011-be2f99e8f519 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:21.697828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-a0bea7fd-e7cf-4ba2-b011-be2f99e8f519 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:09:21.697828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:21.697828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:21.697828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:21.697828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:21.697828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:21.697828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:21.697828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:21.697828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:21.697828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:21.697828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:21.697828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:21.697828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:09:21.697828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:09:21.697828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:09:21.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:09:21.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:21.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:21.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:21.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:21.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:21.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:21.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:21.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:21.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:21.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:21.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:21.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:21.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:21.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:21.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:21.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:21.699654 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:09:21.699654 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:09:21.699654 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:09:21.699654 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:09:21.699654 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:09:21.699654 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:09:21.699654 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:09:21.699654 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:21.699654 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 401/804] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:09:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:21.703860 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:21.704534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bea7fd-e7cf-4ba2-b011-be2f99e8f519 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:21.715608 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 404/805] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 21:09:21.740021 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:21.807908 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bea7fd-e7cf-4ba2-b011-be2f99e8f519 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:21.839939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c537fd-6694-4db7-805a-3454617049ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:21.840538 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c537fd-6694-4db7-805a-3454617049ad None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:21.840816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c537fd-6694-4db7-805a-3454617049ad None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:21.841083 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c537fd-6694-4db7-805a-3454617049ad None admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:21.841747 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c537fd-6694-4db7-805a-3454617049ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:21.842047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c537fd-6694-4db7-805a-3454617049ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:21.843462 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c537fd-6694-4db7-805a-3454617049ad None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:21.855260 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 402/806] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:09:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:21.861405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b5b20-cc94-4138-a8db-aa7107bc043a None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:21.957319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b5b20-cc94-4138-a8db-aa7107bc043a None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:21.974674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c537fd-6694-4db7-805a-3454617049ad None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:21.992330 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b819e-3475-4d41-be78-5a64f410c547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:21.992917 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b819e-3475-4d41-be78-5a64f410c547 None admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:21.993189 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b819e-3475-4d41-be78-5a64f410c547 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:21.993456 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b819e-3475-4d41-be78-5a64f410c547 None admin] PATH_INFO: `/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/12dad9f016a740b98538722e1b723823/roles/d47c18046c97412588d8a6ae24333070` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:21.995540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b819e-3475-4d41-be78-5a64f410c547 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b2c33250ef84d79a4a4742fb4252156, user_id=12dad9f016a740b98538722e1b723823, role_id=d47c18046c97412588d8a6ae24333070)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:21.996985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b819e-3475-4d41-be78-5a64f410c547 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:22.010976 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 405/807] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:09:21 2022] PUT /identity/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/12dad9f016a740b98538722e1b723823/roles/d47c18046c97412588d8a6ae24333070 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:09:22.083408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:22.095375 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c537fd-6694-4db7-805a-3454617049ad None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:22.133341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb59b3e-baaf-46b1-ad7c-54194347a810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:22.134021 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb59b3e-baaf-46b1-ad7c-54194347a810 None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:22.134303 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb59b3e-baaf-46b1-ad7c-54194347a810 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:22.134567 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb59b3e-baaf-46b1-ad7c-54194347a810 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:22.135257 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb59b3e-baaf-46b1-ad7c-54194347a810 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:22.136693 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb59b3e-baaf-46b1-ad7c-54194347a810 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:22.146368 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.sql.core [None req-4cb59b3e-baaf-46b1-ad7c-54194347a810 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 21 21:09:22.146368 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 21 21:09:22.146368 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [parameters: {'id': 'e595d207f3054a078e14e4153ba77686', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 21 21:09:22.146368 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81306) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 21 21:09:22.154398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-4cb59b3e-baaf-46b1-ad7c-54194347a810 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 21 21:09:22.154398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:22.154398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 21:09:22.154398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 21:09:22.154398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 21:09:22.154398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 21:09:22.154398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 21:09:22.154398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 21:09:22.154398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 21:09:22.154398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application conn.query(q) Oct 21 21:09:22.154398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 21:09:22.154398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 21:09:22.154398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 21:09:22.154398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application result.read() Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 21 21:09:22.155287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 21:09:22.160465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application next(self.gen) Oct 21 21:09:22.160465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 21 21:09:22.160465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application yield resource Oct 21 21:09:22.160465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 21:09:22.160465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application next(self.gen) Oct 21 21:09:22.160465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 21 21:09:22.160465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self.session.rollback() Oct 21 21:09:22.160465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 21 21:09:22.160465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self.force_reraise() Oct 21 21:09:22.160465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 21 21:09:22.160465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise self.value Oct 21 21:09:22.160465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 21 21:09:22.160465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 21 21:09:22.160465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 21 21:09:22.160465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application session.commit() Oct 21 21:09:22.160465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 21 21:09:22.160465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 21 21:09:22.161327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 21 21:09:22.161327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._prepare_impl() Oct 21 21:09:22.161327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 21 21:09:22.161327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self.session.flush() Oct 21 21:09:22.161327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 21 21:09:22.161327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._flush(objects) Oct 21 21:09:22.161327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 21 21:09:22.161327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 21 21:09:22.161327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 21 21:09:22.161327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application compat.raise_( Oct 21 21:09:22.161327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 21:09:22.161327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception Oct 21 21:09:22.161327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 21 21:09:22.161327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application flush_context.execute() Oct 21 21:09:22.161327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 21 21:09:22.161327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rec.execute(self) Oct 21 21:09:22.161327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 21 21:09:22.162271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 21 21:09:22.162271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 21 21:09:22.162271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 21 21:09:22.162271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 21 21:09:22.162271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 21 21:09:22.162271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 21 21:09:22.162271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 21 21:09:22.162271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 21 21:09:22.162271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 21 21:09:22.162271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 21 21:09:22.162271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 21 21:09:22.162271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 21 21:09:22.162271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 21 21:09:22.162271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 21 21:09:22.162271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 21 21:09:22.163035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 21:09:22.163035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception Oct 21 21:09:22.163035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 21:09:22.163035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 21:09:22.163035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 21:09:22.163035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 21:09:22.163035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 21:09:22.163035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 21:09:22.163035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 21:09:22.163035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application conn.query(q) Oct 21 21:09:22.163035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 21:09:22.163035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 21:09:22.163035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 21:09:22.163035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application result.read() Oct 21 21:09:22.163035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 21:09:22.163035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 21:09:22.163035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 21:09:22.164151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 21:09:22.164151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 21:09:22.164151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 21:09:22.164151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 21:09:22.164151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 21:09:22.164151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 21 21:09:22.164151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 21 21:09:22.164151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [parameters: {'id': 'e595d207f3054a078e14e4153ba77686', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 21 21:09:22.164151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 21 21:09:22.164151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:22.164151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:22.164151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:22.164151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:22.164151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 21 21:09:22.164151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 21 21:09:22.165082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 21 21:09:22.165082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 21 21:09:22.165082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 21 21:09:22.165082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:22.165082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:09:22.165082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:22.165082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:22.165082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:22.165082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:22.165082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:22.165082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:22.165082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:22.165082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:22.165082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:22.165082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:22.165082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:22.165082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:22.166230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 21 21:09:22.166230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 21 21:09:22.166230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:22.166230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:22.166230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 21 21:09:22.166230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 21 21:09:22.166230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:22.166230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:22.166230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 21 21:09:22.166230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 21 21:09:22.166230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 21 21:09:22.166230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:22.166230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 403/808] 10.222.0.13 () {60 vars in 1137 bytes} [Fri Oct 21 21:09:21 2022] POST /identity/v3/domains => generated 190 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:09:22.167085 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b819e-3475-4d41-be78-5a64f410c547 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:22.229356 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:22.272909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b819e-3475-4d41-be78-5a64f410c547 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:22.305521 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17e318-f656-4305-838a-9163b2d65322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:22.306145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17e318-f656-4305-838a-9163b2d65322 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:22.306419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17e318-f656-4305-838a-9163b2d65322 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:22.306685 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17e318-f656-4305-838a-9163b2d65322 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:22.317075 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17e318-f656-4305-838a-9163b2d65322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:22.318515 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17e318-f656-4305-838a-9163b2d65322 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:22.333483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-7c17e318-f656-4305-838a-9163b2d65322 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:22.333483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:22.333483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:22.333483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:22.333483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:22.333483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:22.333483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:22.333483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:22.333483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:22.333483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:22.333483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:22.333483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:22.333483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:09:22.333483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:09:22.333483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:09:22.333483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:09:22.334409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:22.334409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:22.334409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:22.334409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:22.334409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:22.334409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:22.334409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:22.334409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:22.334409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:22.334409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:22.334409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:22.334409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:22.334409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:22.334409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:22.334409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:22.334409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:22.334409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:09:22.335274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:09:22.335274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:22.335274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:22.335274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 406/809] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:22.340851 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb59b3e-baaf-46b1-ad7c-54194347a810 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:22.442131 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb59b3e-baaf-46b1-ad7c-54194347a810 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:22.475872 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc479f-3558-4967-9621-123d78e89462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:22.476476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc479f-3558-4967-9621-123d78e89462 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:22.476750 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc479f-3558-4967-9621-123d78e89462 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:22.477018 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc479f-3558-4967-9621-123d78e89462 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:22.477706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc479f-3558-4967-9621-123d78e89462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:22.478008 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc479f-3558-4967-9621-123d78e89462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:22.479367 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc479f-3558-4967-9621-123d78e89462 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:22.499654 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 404/810] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:22.932662 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215d7d3-fcac-419a-9c53-c309548d70b4 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:22.932941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215d7d3-fcac-419a-9c53-c309548d70b4 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:22.933216 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215d7d3-fcac-419a-9c53-c309548d70b4 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:22.934554 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 407/811] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:22.940299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d7609-fb94-4825-917f-a8f2cf3c31b9 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:22.940592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d7609-fb94-4825-917f-a8f2cf3c31b9 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:22.940870 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d7609-fb94-4825-917f-a8f2cf3c31b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:22.959121 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-0a6d7609-fb94-4825-917f-a8f2cf3c31b9 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:22.960336 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0a6d7609-fb94-4825-917f-a8f2cf3c31b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:22.969241 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:23.024342 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 405/812] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 21:09:23.291270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215d7d3-fcac-419a-9c53-c309548d70b4 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:23.389745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215d7d3-fcac-419a-9c53-c309548d70b4 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:23.400227 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:23.423731 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257bf1b-e630-4aa8-8121-45062faaaeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:23.424347 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257bf1b-e630-4aa8-8121-45062faaaeae None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:23.424622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257bf1b-e630-4aa8-8121-45062faaaeae None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:23.424891 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257bf1b-e630-4aa8-8121-45062faaaeae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:23.435088 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257bf1b-e630-4aa8-8121-45062faaaeae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:09:23.439915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-4257bf1b-e630-4aa8-8121-45062faaaeae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:09:23.439915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:23.439915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:23.439915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:23.439915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:23.439915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:23.439915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:23.439915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:23.439915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:23.439915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:23.439915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:23.439915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:23.439915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:09:23.439915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:09:23.439915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:09:23.439915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:09:23.441008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:09:23.441008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:09:23.441008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:09:23.441008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:09:23.441008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:23.441008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:23.441008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:23.441008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:23.441008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:23.441008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:23.441008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:23.441008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:23.441008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:23.441008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:23.441008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:23.441008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:23.441852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:23.441852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:23.441852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:23.441852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:23.441852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:09:23.441852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:09:23.441852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:09:23.441852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:09:23.441852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:09:23.441852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:09:23.441852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:09:23.441852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:23.441852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 408/813] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:09:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:23.446974 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d7609-fb94-4825-917f-a8f2cf3c31b9 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:23.543403 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d7609-fb94-4825-917f-a8f2cf3c31b9 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:23.576334 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48a329-0916-4282-873f-8b89f725a654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:23.576939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48a329-0916-4282-873f-8b89f725a654 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:23.577215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48a329-0916-4282-873f-8b89f725a654 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:23.577482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48a329-0916-4282-873f-8b89f725a654 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:23.578130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48a329-0916-4282-873f-8b89f725a654 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:23.578948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48a329-0916-4282-873f-8b89f725a654 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:23.580342 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48a329-0916-4282-873f-8b89f725a654 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:23.588306 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 406/814] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:09:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:23.594907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257bf1b-e630-4aa8-8121-45062faaaeae None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:23.691336 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257bf1b-e630-4aa8-8121-45062faaaeae None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:23.724233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50feb1-99b4-451d-9c89-011daa1b9fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:23.724864 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50feb1-99b4-451d-9c89-011daa1b9fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:23.725144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50feb1-99b4-451d-9c89-011daa1b9fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:23.725407 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50feb1-99b4-451d-9c89-011daa1b9fe4 None admin] PATH_INFO: `/v3/users/bd377bb9e16f45a0a027ccdb66acf7b9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:23.739249 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50feb1-99b4-451d-9c89-011daa1b9fe4 None admin] RBAC: Authorizing `identity:get_user(user_id=bd377bb9e16f45a0a027ccdb66acf7b9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:23.740721 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50feb1-99b4-451d-9c89-011daa1b9fe4 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:23.742364 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 409/815] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:09:23 2022] GET /identity/v3/users/bd377bb9e16f45a0a027ccdb66acf7b9 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:23.748460 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48a329-0916-4282-873f-8b89f725a654 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:23.758035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:23.846714 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48a329-0916-4282-873f-8b89f725a654 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:23.878760 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79780a54-e3c2-4eb2-aaf1-caf3db1d5b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:23.879379 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79780a54-e3c2-4eb2-aaf1-caf3db1d5b77 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:23.879693 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79780a54-e3c2-4eb2-aaf1-caf3db1d5b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:23.879963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79780a54-e3c2-4eb2-aaf1-caf3db1d5b77 None admin] PATH_INFO: `/v3/projects/6a3c5d299af94cca9e9214a3ec7508a9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:23.881058 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79780a54-e3c2-4eb2-aaf1-caf3db1d5b77 None admin] RBAC: Authorizing `identity:get_project(project_id=6a3c5d299af94cca9e9214a3ec7508a9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:23.882484 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79780a54-e3c2-4eb2-aaf1-caf3db1d5b77 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:23.884290 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 407/816] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:09:23 2022] GET /identity/v3/projects/6a3c5d299af94cca9e9214a3ec7508a9 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:23.890254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50feb1-99b4-451d-9c89-011daa1b9fe4 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:23.918249 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:23.988291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50feb1-99b4-451d-9c89-011daa1b9fe4 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:24.021083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8f2e7-d684-43ea-98b3-89c792c19575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:24.021735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8f2e7-d684-43ea-98b3-89c792c19575 None admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:24.022012 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8f2e7-d684-43ea-98b3-89c792c19575 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:24.022282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8f2e7-d684-43ea-98b3-89c792c19575 None admin] PATH_INFO: `/v3/projects/6a3c5d299af94cca9e9214a3ec7508a9/users/bd377bb9e16f45a0a027ccdb66acf7b9/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:24.024174 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8f2e7-d684-43ea-98b3-89c792c19575 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a3c5d299af94cca9e9214a3ec7508a9, user_id=bd377bb9e16f45a0a027ccdb66acf7b9, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:24.025579 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8f2e7-d684-43ea-98b3-89c792c19575 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:24.038046 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 410/817] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:09:23 2022] PUT /identity/v3/projects/6a3c5d299af94cca9e9214a3ec7508a9/users/bd377bb9e16f45a0a027ccdb66acf7b9/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:09:24.177259 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36ed68-8109-44dd-8873-4371ffd32b39 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:24.177770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36ed68-8109-44dd-8873-4371ffd32b39 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:24.177770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36ed68-8109-44dd-8873-4371ffd32b39 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:24.179161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 408/818] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:24.184957 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571b537-96c9-41b3-9e57-794adf667fef None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:24.185245 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571b537-96c9-41b3-9e57-794adf667fef None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:24.185523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571b537-96c9-41b3-9e57-794adf667fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:24.204462 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-6571b537-96c9-41b3-9e57-794adf667fef None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:24.205680 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-6571b537-96c9-41b3-9e57-794adf667fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:24.271482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 411/819] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:24.537073 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36ed68-8109-44dd-8873-4371ffd32b39 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:24.611432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:24.636724 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36ed68-8109-44dd-8873-4371ffd32b39 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:24.668928 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625376a-4476-46a2-abb3-fa7fc5b28023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:24.669548 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625376a-4476-46a2-abb3-fa7fc5b28023 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:24.669821 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625376a-4476-46a2-abb3-fa7fc5b28023 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:24.670086 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625376a-4476-46a2-abb3-fa7fc5b28023 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:24.680167 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625376a-4476-46a2-abb3-fa7fc5b28023 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:09:24.684816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-7625376a-4476-46a2-abb3-fa7fc5b28023 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 21 21:09:24.684816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:24.684816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:24.684816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:24.684816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:24.684816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:24.684816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:24.684816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:24.684816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:24.684816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:24.684816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:24.684816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:24.684816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:09:24.684816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:09:24.684816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:09:24.684816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:09:24.685846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:09:24.685846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:09:24.685846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:09:24.685846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:09:24.685846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:24.685846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:24.685846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:24.685846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:24.685846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:24.685846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:24.685846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:24.685846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:24.685846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:24.685846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:24.685846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:24.685846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:24.687046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:24.687046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:24.687046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:24.687046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:24.687046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:09:24.687046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:09:24.687046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:09:24.687046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:09:24.687046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:09:24.687046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:09:24.687046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 21 21:09:24.687046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:24.687046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 409/820] 10.222.0.13 () {56 vars in 1120 bytes} [Fri Oct 21 21:09:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:24.692376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571b537-96c9-41b3-9e57-794adf667fef None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:24.710667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:24.794483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571b537-96c9-41b3-9e57-794adf667fef None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:24.827379 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3848a4f-523a-42ee-b739-6efba4b7100f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:24.828010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3848a4f-523a-42ee-b739-6efba4b7100f None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:24.828292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3848a4f-523a-42ee-b739-6efba4b7100f None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:24.828566 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3848a4f-523a-42ee-b739-6efba4b7100f None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:24.829219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3848a4f-523a-42ee-b739-6efba4b7100f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:24.829521 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3848a4f-523a-42ee-b739-6efba4b7100f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:24.830880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3848a4f-523a-42ee-b739-6efba4b7100f None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:24.838612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 412/821] 10.222.0.13 () {56 vars in 1115 bytes} [Fri Oct 21 21:09:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:24.844228 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625376a-4476-46a2-abb3-fa7fc5b28023 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:24.942510 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625376a-4476-46a2-abb3-fa7fc5b28023 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:24.974229 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd6ff1-1928-4d73-b1a5-40773567dc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:24.974849 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd6ff1-1928-4d73-b1a5-40773567dc8f None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:24.975128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd6ff1-1928-4d73-b1a5-40773567dc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:24.975400 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd6ff1-1928-4d73-b1a5-40773567dc8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:24.977247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:24.987084 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd6ff1-1928-4d73-b1a5-40773567dc8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:24.988573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd6ff1-1928-4d73-b1a5-40773567dc8f None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:25.002695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-64dd6ff1-1928-4d73-b1a5-40773567dc8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:25.002695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:25.002695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:25.002695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:25.002695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:25.002695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:25.002695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:25.002695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:25.002695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:25.002695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:25.002695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:25.002695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:25.002695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:09:25.002695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:09:25.002695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:09:25.002695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:09:25.003805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:25.003805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:25.003805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:25.003805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:25.003805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:25.003805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:25.003805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:25.003805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:25.003805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:25.003805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:25.003805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:25.003805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:25.003805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:25.003805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:25.003805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:25.003805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:25.003805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:09:25.004853 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:09:25.004853 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:25.004853 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:25.004853 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 410/822] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:25.009698 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3848a4f-523a-42ee-b739-6efba4b7100f None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:25.087918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:25.106380 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3848a4f-523a-42ee-b739-6efba4b7100f None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:25.138819 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2a083-3656-4a2c-b9fb-4dd7220e8e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:25.139421 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2a083-3656-4a2c-b9fb-4dd7220e8e50 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:25.139729 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2a083-3656-4a2c-b9fb-4dd7220e8e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:25.140007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2a083-3656-4a2c-b9fb-4dd7220e8e50 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:25.140647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2a083-3656-4a2c-b9fb-4dd7220e8e50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:25.141009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2a083-3656-4a2c-b9fb-4dd7220e8e50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:25.142362 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2a083-3656-4a2c-b9fb-4dd7220e8e50 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:25.153551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 413/823] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:25.160028 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd6ff1-1928-4d73-b1a5-40773567dc8f None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:25.255501 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd6ff1-1928-4d73-b1a5-40773567dc8f None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:25.287791 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d33932-095f-4f49-9469-1cd227717a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:25.288413 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d33932-095f-4f49-9469-1cd227717a31 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:25.288692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d33932-095f-4f49-9469-1cd227717a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:25.288968 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d33932-095f-4f49-9469-1cd227717a31 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:25.296140 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d33932-095f-4f49-9469-1cd227717a31 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:25.298135 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d33932-095f-4f49-9469-1cd227717a31 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:25.308892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-d3d33932-095f-4f49-9469-1cd227717a31 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 21 21:09:25.308892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:25.308892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:25.308892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:25.308892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:25.308892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:25.308892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:25.308892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:25.308892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:25.308892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:25.308892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:25.308892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:25.308892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:09:25.308892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:09:25.308892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:09:25.308892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:25.310003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:25.311069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:25.311069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:25.311069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:09:25.311069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:09:25.311069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:09:25.311069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:09:25.311069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:09:25.311069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:09:25.311069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 21 21:09:25.311069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:25.314851 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 411/824] 10.222.0.13 () {56 vars in 1093 bytes} [Fri Oct 21 21:09:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:25.316071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2a083-3656-4a2c-b9fb-4dd7220e8e50 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:25.411391 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2a083-3656-4a2c-b9fb-4dd7220e8e50 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:25.445222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb675ed-1c8d-4909-b8a8-2420bf927b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:25.445823 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb675ed-1c8d-4909-b8a8-2420bf927b1f None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:25.446151 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb675ed-1c8d-4909-b8a8-2420bf927b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:25.446428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb675ed-1c8d-4909-b8a8-2420bf927b1f None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:25.447144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb675ed-1c8d-4909-b8a8-2420bf927b1f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:25.447453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb675ed-1c8d-4909-b8a8-2420bf927b1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:25.448898 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb675ed-1c8d-4909-b8a8-2420bf927b1f None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:25.457406 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:25.466658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 414/825] 10.222.0.13 () {56 vars in 1133 bytes} [Fri Oct 21 21:09:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:25.472844 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d33932-095f-4f49-9469-1cd227717a31 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:25.518116 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:25.569949 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d33932-095f-4f49-9469-1cd227717a31 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:25.602716 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5200a-8bff-422a-b7e4-0aabf5bc4aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:25.603326 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5200a-8bff-422a-b7e4-0aabf5bc4aaf None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:25.603642 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5200a-8bff-422a-b7e4-0aabf5bc4aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:25.603921 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5200a-8bff-422a-b7e4-0aabf5bc4aaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:25.667937 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5200a-8bff-422a-b7e4-0aabf5bc4aaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:25.669334 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5200a-8bff-422a-b7e4-0aabf5bc4aaf None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:25.711639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:25.768000 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 412/826] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:09:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 21 21:09:25.773886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb675ed-1c8d-4909-b8a8-2420bf927b1f None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:25.868864 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb675ed-1c8d-4909-b8a8-2420bf927b1f None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:25.902198 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113af281-800d-48d6-b490-7017735e9ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:25.902824 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113af281-800d-48d6-b490-7017735e9ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:25.903103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113af281-800d-48d6-b490-7017735e9ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:25.903372 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113af281-800d-48d6-b490-7017735e9ce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:25.912991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113af281-800d-48d6-b490-7017735e9ce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:25.914408 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113af281-800d-48d6-b490-7017735e9ce5 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:25.928400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-113af281-800d-48d6-b490-7017735e9ce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:25.928400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:25.928400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:25.928400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:25.928400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:25.928400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:25.928400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:25.928400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:25.928400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:25.928400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:25.928400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:25.928400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:25.928400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:09:25.928400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:09:25.928400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:09:25.928400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:09:25.929441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:25.929441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:25.929441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:25.929441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:25.929441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:25.929441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:25.929441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:25.929441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:25.929441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:25.929441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:25.929441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:25.929441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:25.929441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:25.929441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:25.929441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:25.929441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:25.929441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:09:25.930365 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:09:25.930365 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:09:25.930365 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:25.930365 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 415/827] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:25.935204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5200a-8bff-422a-b7e4-0aabf5bc4aaf None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:26.030854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5200a-8bff-422a-b7e4-0aabf5bc4aaf None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:26.067645 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195d958-eb68-487a-9213-d4936e519d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:26.068256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195d958-eb68-487a-9213-d4936e519d9d None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:26.068532 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195d958-eb68-487a-9213-d4936e519d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:26.068802 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195d958-eb68-487a-9213-d4936e519d9d None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:26.069439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195d958-eb68-487a-9213-d4936e519d9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:26.069741 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195d958-eb68-487a-9213-d4936e519d9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:26.071114 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195d958-eb68-487a-9213-d4936e519d9d None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:26.082215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 413/828] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:26.087992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113af281-800d-48d6-b490-7017735e9ce5 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:26.136731 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:26.151900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2486ec-bf04-49e2-a009-91586a8dfde8 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:26.152190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2486ec-bf04-49e2-a009-91586a8dfde8 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:26.152470 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2486ec-bf04-49e2-a009-91586a8dfde8 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:26.153811 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 414/829] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:26.159431 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4767a4-8337-462c-9b2e-d5a4776a1ce2 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:26.159753 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4767a4-8337-462c-9b2e-d5a4776a1ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:26.160034 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4767a4-8337-462c-9b2e-d5a4776a1ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:26.173043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:26.178610 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-aa4767a4-8337-462c-9b2e-d5a4776a1ce2 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:26.179787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-aa4767a4-8337-462c-9b2e-d5a4776a1ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:26.188271 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113af281-800d-48d6-b490-7017735e9ce5 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:26.220619 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed90464-04af-446d-ac93-f10b737d1db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:26.221240 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed90464-04af-446d-ac93-f10b737d1db5 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:26.221521 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed90464-04af-446d-ac93-f10b737d1db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:26.221795 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed90464-04af-446d-ac93-f10b737d1db5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:26.228392 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed90464-04af-446d-ac93-f10b737d1db5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:26.229813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed90464-04af-446d-ac93-f10b737d1db5 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:26.241724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-eed90464-04af-446d-ac93-f10b737d1db5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:09:26.241724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:26.241724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:26.241724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:26.241724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:26.241724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:26.241724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:26.241724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:26.241724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:26.241724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:26.241724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:26.241724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:26.241724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:09:26.241724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:09:26.241724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:09:26.242644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:09:26.242644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:26.242644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:26.242644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:26.242644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:26.242644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:26.242644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:26.242644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:26.242644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:26.242644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:26.242644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:26.242644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:26.242644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:26.242644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:26.242644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:26.242644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:26.242644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:26.243531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:09:26.243531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:09:26.243531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:09:26.243531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:09:26.243531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:09:26.243531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:09:26.243531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:09:26.243531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:26.243531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 416/830] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:09:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:26.247604 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 415/831] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 21:09:26.248637 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed90464-04af-446d-ac93-f10b737d1db5 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:26.342376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed90464-04af-446d-ac93-f10b737d1db5 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:26.352185 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:26.374989 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e6cf7-91d9-4e8b-8e5c-12097d67c9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:26.376167 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e6cf7-91d9-4e8b-8e5c-12097d67c9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:26.376447 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e6cf7-91d9-4e8b-8e5c-12097d67c9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:26.376715 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e6cf7-91d9-4e8b-8e5c-12097d67c9f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:26.377387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e6cf7-91d9-4e8b-8e5c-12097d67c9f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:26.377688 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e6cf7-91d9-4e8b-8e5c-12097d67c9f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:26.379082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e6cf7-91d9-4e8b-8e5c-12097d67c9f4 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:26.390562 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 417/832] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:09:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:26.396648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4767a4-8337-462c-9b2e-d5a4776a1ce2 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:26.491041 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4767a4-8337-462c-9b2e-d5a4776a1ce2 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:26.514074 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e6cf7-91d9-4e8b-8e5c-12097d67c9f4 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:26.524458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd99bc8-355e-48ad-b7c7-c1cf3d14b108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:26.525063 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd99bc8-355e-48ad-b7c7-c1cf3d14b108 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:26.525395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd99bc8-355e-48ad-b7c7-c1cf3d14b108 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:26.525663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd99bc8-355e-48ad-b7c7-c1cf3d14b108 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:26.526393 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd99bc8-355e-48ad-b7c7-c1cf3d14b108 None admin] RBAC: Adding query filter params (role.id=d47c18046c97412588d8a6ae24333070, scope.project.id=8b2c33250ef84d79a4a4742fb4252156, user.id=12dad9f016a740b98538722e1b723823) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:26.526721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd99bc8-355e-48ad-b7c7-c1cf3d14b108 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:26.528189 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd99bc8-355e-48ad-b7c7-c1cf3d14b108 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:26.530855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:26.534521 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 416/833] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:09:26 2022] GET /identity/v3/role_assignments?user.id=12dad9f016a740b98538722e1b723823&scope.project.id=8b2c33250ef84d79a4a4742fb4252156&role.id=d47c18046c97412588d8a6ae24333070 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:26.618986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e6cf7-91d9-4e8b-8e5c-12097d67c9f4 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:26.654893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1beba1-2f50-44f7-9e65-b5f82d1aca5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:26.655537 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1beba1-2f50-44f7-9e65-b5f82d1aca5e None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:26.655853 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1beba1-2f50-44f7-9e65-b5f82d1aca5e None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:26.656122 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1beba1-2f50-44f7-9e65-b5f82d1aca5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:26.667396 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1beba1-2f50-44f7-9e65-b5f82d1aca5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:09:26.672886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-2d1beba1-2f50-44f7-9e65-b5f82d1aca5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:09:26.672886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:26.672886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:26.672886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:26.672886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:26.672886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:26.672886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:26.672886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:26.672886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:26.672886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:26.672886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:26.672886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:26.672886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:09:26.672886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:09:26.672886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:09:26.672886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:09:26.673888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:09:26.673888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:09:26.673888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:09:26.673888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:09:26.673888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:26.673888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:26.673888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:26.673888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:26.673888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:26.673888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:26.673888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:26.673888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:26.673888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:26.673888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:26.673888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:26.673888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:26.674712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:26.674712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:26.674712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:26.674712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:26.674712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:09:26.674712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:09:26.674712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:09:26.674712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:09:26.674712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:09:26.674712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:09:26.674712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:09:26.674712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:26.674712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 418/834] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:09:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:09:26.680247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd99bc8-355e-48ad-b7c7-c1cf3d14b108 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:26.779858 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd99bc8-355e-48ad-b7c7-c1cf3d14b108 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:26.813172 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02b6fb-6e06-4d95-ac03-e4f41e96dbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:26.813759 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02b6fb-6e06-4d95-ac03-e4f41e96dbba None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:26.814030 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02b6fb-6e06-4d95-ac03-e4f41e96dbba None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:26.814294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02b6fb-6e06-4d95-ac03-e4f41e96dbba None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:26.814938 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02b6fb-6e06-4d95-ac03-e4f41e96dbba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:26.815235 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02b6fb-6e06-4d95-ac03-e4f41e96dbba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:26.816622 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02b6fb-6e06-4d95-ac03-e4f41e96dbba None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:26.825225 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 417/835] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:09:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:26.831142 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1beba1-2f50-44f7-9e65-b5f82d1aca5e None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:26.861952 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:26.931997 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1beba1-2f50-44f7-9e65-b5f82d1aca5e None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:26.964485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5ab6a-ecf1-4937-9d98-25f314a9ed9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:26.965109 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5ab6a-ecf1-4937-9d98-25f314a9ed9e None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:26.965386 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5ab6a-ecf1-4937-9d98-25f314a9ed9e None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:26.965665 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5ab6a-ecf1-4937-9d98-25f314a9ed9e None admin] PATH_INFO: `/v3/users/bd377bb9e16f45a0a027ccdb66acf7b9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:26.966787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b5ab6a-ecf1-4937-9d98-25f314a9ed9e None admin] RBAC: Authorizing `identity:get_user(user_id=bd377bb9e16f45a0a027ccdb66acf7b9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:26.968246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b5ab6a-ecf1-4937-9d98-25f314a9ed9e None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:26.970410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 419/836] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:09:26 2022] GET /identity/v3/users/bd377bb9e16f45a0a027ccdb66acf7b9 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:26.976239 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02b6fb-6e06-4d95-ac03-e4f41e96dbba None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:27.037905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:27.074474 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02b6fb-6e06-4d95-ac03-e4f41e96dbba None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:27.108369 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc71c9-e4ef-4ded-b1fe-2d38671ebc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:27.108979 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc71c9-e4ef-4ded-b1fe-2d38671ebc12 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:27.109254 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc71c9-e4ef-4ded-b1fe-2d38671ebc12 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:27.109553 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc71c9-e4ef-4ded-b1fe-2d38671ebc12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:27.180408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc71c9-e4ef-4ded-b1fe-2d38671ebc12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:27.181881 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc71c9-e4ef-4ded-b1fe-2d38671ebc12 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:27.239373 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:27.279851 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 418/837] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:09:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 21:09:27.286393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5ab6a-ecf1-4937-9d98-25f314a9ed9e None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:27.381622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:27.385440 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5ab6a-ecf1-4937-9d98-25f314a9ed9e None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:27.418145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1b0a7-8cce-4480-8e6c-479d24259eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:27.418758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1b0a7-8cce-4480-8e6c-479d24259eca None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:27.419035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1b0a7-8cce-4480-8e6c-479d24259eca None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:27.419302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1b0a7-8cce-4480-8e6c-479d24259eca None admin] PATH_INFO: `/v3/projects/6a3c5d299af94cca9e9214a3ec7508a9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:27.420426 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1b0a7-8cce-4480-8e6c-479d24259eca None admin] RBAC: Authorizing `identity:get_project(project_id=6a3c5d299af94cca9e9214a3ec7508a9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:27.421822 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1b0a7-8cce-4480-8e6c-479d24259eca None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:27.423446 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 420/838] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:09:27 2022] GET /identity/v3/projects/6a3c5d299af94cca9e9214a3ec7508a9 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:27.429526 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc71c9-e4ef-4ded-b1fe-2d38671ebc12 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:27.525783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc71c9-e4ef-4ded-b1fe-2d38671ebc12 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:27.558872 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e0ce1-7cd9-4974-8931-2ccda0e470a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:27.559460 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e0ce1-7cd9-4974-8931-2ccda0e470a7 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:27.559764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e0ce1-7cd9-4974-8931-2ccda0e470a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:27.560031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e0ce1-7cd9-4974-8931-2ccda0e470a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:27.560733 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e0ce1-7cd9-4974-8931-2ccda0e470a7 None admin] RBAC: Adding query filter params (role.id=c514b065b7604094b899c586f79b0c5b, scope.project.id=6a3c5d299af94cca9e9214a3ec7508a9, user.id=bd377bb9e16f45a0a027ccdb66acf7b9) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:27.561036 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e0ce1-7cd9-4974-8931-2ccda0e470a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:27.562465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e0ce1-7cd9-4974-8931-2ccda0e470a7 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:27.568403 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 419/839] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:09:27 2022] GET /identity/v3/role_assignments?user.id=bd377bb9e16f45a0a027ccdb66acf7b9&scope.project.id=6a3c5d299af94cca9e9214a3ec7508a9&role.id=c514b065b7604094b899c586f79b0c5b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:29.692317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2018c82-04ba-4ab2-ac2d-20a2083d71b8 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:29.692910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2018c82-04ba-4ab2-ac2d-20a2083d71b8 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:29.692910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2018c82-04ba-4ab2-ac2d-20a2083d71b8 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:29.694224 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 421/840] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:29.699486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425f86d-72a3-41ee-8c80-45b30690b522 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:29.699913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425f86d-72a3-41ee-8c80-45b30690b522 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:29.700060 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425f86d-72a3-41ee-8c80-45b30690b522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:29.718069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-c425f86d-72a3-41ee-8c80-45b30690b522 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:29.719265 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-c425f86d-72a3-41ee-8c80-45b30690b522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:29.738407 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:29.783715 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 420/841] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 21:09:30.053487 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2018c82-04ba-4ab2-ac2d-20a2083d71b8 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:30.150942 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2018c82-04ba-4ab2-ac2d-20a2083d71b8 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:30.182506 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f47430-e075-446e-9de1-d4dcad5fe247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:30.183116 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f47430-e075-446e-9de1-d4dcad5fe247 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:30.183395 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f47430-e075-446e-9de1-d4dcad5fe247 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:30.183699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f47430-e075-446e-9de1-d4dcad5fe247 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:30.184795 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f47430-e075-446e-9de1-d4dcad5fe247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:30.186197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f47430-e075-446e-9de1-d4dcad5fe247 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:30.188287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 422/842] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:30.193861 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425f86d-72a3-41ee-8c80-45b30690b522 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:30.288019 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425f86d-72a3-41ee-8c80-45b30690b522 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:30.320331 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53935e2-8828-4453-9eeb-d78814b6421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:30.320870 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53935e2-8828-4453-9eeb-d78814b6421e None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:30.321148 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53935e2-8828-4453-9eeb-d78814b6421e None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:30.321419 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53935e2-8828-4453-9eeb-d78814b6421e None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:30.322058 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53935e2-8828-4453-9eeb-d78814b6421e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:30.323479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53935e2-8828-4453-9eeb-d78814b6421e None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:30.344409 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 421/843] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:09:30 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:09:32.472309 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6155b-65c5-41d8-a7ba-544dea67051a None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:32.472860 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6155b-65c5-41d8-a7ba-544dea67051a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:32.472860 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6155b-65c5-41d8-a7ba-544dea67051a None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:32.474223 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 423/844] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:32.479484 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665dc69-3ef8-4c97-ac6b-b0f220585319 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:32.480035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665dc69-3ef8-4c97-ac6b-b0f220585319 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:32.480145 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665dc69-3ef8-4c97-ac6b-b0f220585319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:32.497756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-3665dc69-3ef8-4c97-ac6b-b0f220585319 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:32.498919 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-3665dc69-3ef8-4c97-ac6b-b0f220585319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:32.506048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:32.563686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 422/845] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:32.835668 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6155b-65c5-41d8-a7ba-544dea67051a None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:32.870895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:32.935260 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6155b-65c5-41d8-a7ba-544dea67051a None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:32.966963 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b252ec0-5470-475c-a0d4-b4170b9a2984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:32.967597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b252ec0-5470-475c-a0d4-b4170b9a2984 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:32.967882 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b252ec0-5470-475c-a0d4-b4170b9a2984 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:32.968158 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b252ec0-5470-475c-a0d4-b4170b9a2984 None admin] PATH_INFO: `/v3/roles/01dbc90c2ce041afa161e621bd524341` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:32.970057 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b252ec0-5470-475c-a0d4-b4170b9a2984 None admin] RBAC: Authorizing `identity:get_role(role_id=01dbc90c2ce041afa161e621bd524341)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:32.971407 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b252ec0-5470-475c-a0d4-b4170b9a2984 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:32.972744 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 424/846] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:09:32 2022] GET /identity/v3/roles/01dbc90c2ce041afa161e621bd524341 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:32.978540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665dc69-3ef8-4c97-ac6b-b0f220585319 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:33.072805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665dc69-3ef8-4c97-ac6b-b0f220585319 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:33.105212 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377fb66-ecfd-4424-a026-4f69f8dd5867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:33.105827 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377fb66-ecfd-4424-a026-4f69f8dd5867 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:33.106105 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377fb66-ecfd-4424-a026-4f69f8dd5867 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:33.106378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377fb66-ecfd-4424-a026-4f69f8dd5867 None admin] PATH_INFO: `/v3/users/16f2bf71d48342ebbf1855c4f930ad95` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:33.117337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:33.119934 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377fb66-ecfd-4424-a026-4f69f8dd5867 None admin] RBAC: Authorizing `identity:get_user(user_id=16f2bf71d48342ebbf1855c4f930ad95)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:33.121706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377fb66-ecfd-4424-a026-4f69f8dd5867 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:33.123494 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 423/847] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:09:32 2022] GET /identity/v3/users/16f2bf71d48342ebbf1855c4f930ad95 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:33.128945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b252ec0-5470-475c-a0d4-b4170b9a2984 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:33.223519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b252ec0-5470-475c-a0d4-b4170b9a2984 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:33.225005 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:33.257006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740219a-a541-4d9a-ba26-b2229662d816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:33.257615 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740219a-a541-4d9a-ba26-b2229662d816 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:33.257896 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740219a-a541-4d9a-ba26-b2229662d816 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:33.258168 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740219a-a541-4d9a-ba26-b2229662d816 None admin] PATH_INFO: `/v3/projects/6a3c5d299af94cca9e9214a3ec7508a9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:33.259277 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9740219a-a541-4d9a-ba26-b2229662d816 None admin] RBAC: Authorizing `identity:get_project(project_id=6a3c5d299af94cca9e9214a3ec7508a9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:33.260715 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9740219a-a541-4d9a-ba26-b2229662d816 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:33.262562 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 425/848] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:09:33 2022] GET /identity/v3/projects/6a3c5d299af94cca9e9214a3ec7508a9 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:33.268382 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377fb66-ecfd-4424-a026-4f69f8dd5867 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:33.361277 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377fb66-ecfd-4424-a026-4f69f8dd5867 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:33.392921 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44d25f-a143-4135-8629-b211f0600650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:33.393550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44d25f-a143-4135-8629-b211f0600650 None admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:33.393824 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44d25f-a143-4135-8629-b211f0600650 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:33.394094 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44d25f-a143-4135-8629-b211f0600650 None admin] PATH_INFO: `/v3/projects/6a3c5d299af94cca9e9214a3ec7508a9/users/16f2bf71d48342ebbf1855c4f930ad95/roles/01dbc90c2ce041afa161e621bd524341` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:33.396177 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c44d25f-a143-4135-8629-b211f0600650 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a3c5d299af94cca9e9214a3ec7508a9, user_id=16f2bf71d48342ebbf1855c4f930ad95, role_id=01dbc90c2ce041afa161e621bd524341)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:33.397585 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c44d25f-a143-4135-8629-b211f0600650 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:33.410411 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 424/849] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:09:33 2022] PUT /identity/v3/projects/6a3c5d299af94cca9e9214a3ec7508a9/users/16f2bf71d48342ebbf1855c4f930ad95/roles/01dbc90c2ce041afa161e621bd524341 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:09:35.515711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027374c-23ea-4d8d-b893-671f6c144ba5 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:35.516259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027374c-23ea-4d8d-b893-671f6c144ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:35.516259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027374c-23ea-4d8d-b893-671f6c144ba5 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:35.517582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 426/850] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:35.522795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7aab2d-8785-4636-ad2c-80449ec49b81 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:35.523209 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7aab2d-8785-4636-ad2c-80449ec49b81 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:35.523337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7aab2d-8785-4636-ad2c-80449ec49b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:35.540833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-2d7aab2d-8785-4636-ad2c-80449ec49b81 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:35.542012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2d7aab2d-8785-4636-ad2c-80449ec49b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:35.594795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:35.607030 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 425/851] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:35.873001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027374c-23ea-4d8d-b893-671f6c144ba5 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:35.971740 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027374c-23ea-4d8d-b893-671f6c144ba5 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:36.003651 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5661e-bc1f-4533-867f-1c6615709f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:36.004261 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5661e-bc1f-4533-867f-1c6615709f51 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:36.004536 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5661e-bc1f-4533-867f-1c6615709f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:36.004804 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5661e-bc1f-4533-867f-1c6615709f51 None admin] PATH_INFO: `/v3/roles/01dbc90c2ce041afa161e621bd524341` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:36.006155 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5661e-bc1f-4533-867f-1c6615709f51 None admin] RBAC: Authorizing `identity:get_role(role_id=01dbc90c2ce041afa161e621bd524341)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:36.007505 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5661e-bc1f-4533-867f-1c6615709f51 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:36.008837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 427/852] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:09:35 2022] GET /identity/v3/roles/01dbc90c2ce041afa161e621bd524341 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:36.014838 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7aab2d-8785-4636-ad2c-80449ec49b81 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:36.107784 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7aab2d-8785-4636-ad2c-80449ec49b81 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:36.139477 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb195634-fd9f-4e20-900c-c6287e1fdca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:36.140048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb195634-fd9f-4e20-900c-c6287e1fdca0 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:36.140837 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb195634-fd9f-4e20-900c-c6287e1fdca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:36.141105 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb195634-fd9f-4e20-900c-c6287e1fdca0 None admin] PATH_INFO: `/v3/users/16f2bf71d48342ebbf1855c4f930ad95` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:36.142195 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb195634-fd9f-4e20-900c-c6287e1fdca0 None admin] RBAC: Authorizing `identity:get_user(user_id=16f2bf71d48342ebbf1855c4f930ad95)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:36.143650 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb195634-fd9f-4e20-900c-c6287e1fdca0 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:36.145184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 426/853] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:09:36 2022] GET /identity/v3/users/16f2bf71d48342ebbf1855c4f930ad95 => generated 286 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:36.150639 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5661e-bc1f-4533-867f-1c6615709f51 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:36.302592 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:36.359982 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5661e-bc1f-4533-867f-1c6615709f51 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:36.392646 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a639f9-ea07-4435-8980-045126f88a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:36.393177 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a639f9-ea07-4435-8980-045126f88a42 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:36.393452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a639f9-ea07-4435-8980-045126f88a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:36.393719 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a639f9-ea07-4435-8980-045126f88a42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:36.455865 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a639f9-ea07-4435-8980-045126f88a42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:36.457277 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a639f9-ea07-4435-8980-045126f88a42 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:36.493246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:36.550886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 428/854] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:09:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 21:09:36.556955 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb195634-fd9f-4e20-900c-c6287e1fdca0 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:36.612680 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:36.651480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb195634-fd9f-4e20-900c-c6287e1fdca0 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:36.684068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f9f47-2e47-48d7-b440-1c366469605b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:36.684678 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f9f47-2e47-48d7-b440-1c366469605b None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:36.684954 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f9f47-2e47-48d7-b440-1c366469605b None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:36.685224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f9f47-2e47-48d7-b440-1c366469605b None admin] PATH_INFO: `/v3/projects/6a3c5d299af94cca9e9214a3ec7508a9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:36.686316 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f9f47-2e47-48d7-b440-1c366469605b None admin] RBAC: Authorizing `identity:get_project(project_id=6a3c5d299af94cca9e9214a3ec7508a9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:36.687784 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f9f47-2e47-48d7-b440-1c366469605b None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:36.689386 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 427/855] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:09:36 2022] GET /identity/v3/projects/6a3c5d299af94cca9e9214a3ec7508a9 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:36.695530 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a639f9-ea07-4435-8980-045126f88a42 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:36.790232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a639f9-ea07-4435-8980-045126f88a42 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:36.821162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8a87d-378f-4679-a946-90936a88262c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:36.821752 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8a87d-378f-4679-a946-90936a88262c None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:36.822030 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8a87d-378f-4679-a946-90936a88262c None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:36.822297 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8a87d-378f-4679-a946-90936a88262c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:36.823013 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f8a87d-378f-4679-a946-90936a88262c None admin] RBAC: Adding query filter params (role.id=01dbc90c2ce041afa161e621bd524341, scope.project.id=6a3c5d299af94cca9e9214a3ec7508a9, user.id=16f2bf71d48342ebbf1855c4f930ad95) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:36.823319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f8a87d-378f-4679-a946-90936a88262c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:36.824750 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f8a87d-378f-4679-a946-90936a88262c None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:36.830732 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 429/856] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:09:36 2022] GET /identity/v3/role_assignments?user.id=16f2bf71d48342ebbf1855c4f930ad95&scope.project.id=6a3c5d299af94cca9e9214a3ec7508a9&role.id=01dbc90c2ce041afa161e621bd524341 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:38.951952 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7adf9-c810-4750-b242-8dcccaeb895c None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:38.952498 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7adf9-c810-4750-b242-8dcccaeb895c None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:38.952498 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7adf9-c810-4750-b242-8dcccaeb895c None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:38.953883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 428/857] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:38.959106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5649be9-9df5-4588-b199-09d0db755ec5 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:38.959527 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5649be9-9df5-4588-b199-09d0db755ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:38.959688 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5649be9-9df5-4588-b199-09d0db755ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:38.965854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:38.978746 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-e5649be9-9df5-4588-b199-09d0db755ec5 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:38.979954 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-e5649be9-9df5-4588-b199-09d0db755ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:39.042978 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 430/858] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:39.313525 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7adf9-c810-4750-b242-8dcccaeb895c None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:39.410668 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7adf9-c810-4750-b242-8dcccaeb895c None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:39.435047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:39.444053 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455dd356-f43c-4381-bd71-c76c0608667c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:39.444678 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455dd356-f43c-4381-bd71-c76c0608667c None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:39.444953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455dd356-f43c-4381-bd71-c76c0608667c None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:39.445225 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455dd356-f43c-4381-bd71-c76c0608667c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:39.446311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455dd356-f43c-4381-bd71-c76c0608667c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:39.447774 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455dd356-f43c-4381-bd71-c76c0608667c None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:39.449354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 429/859] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:39.455743 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5649be9-9df5-4588-b199-09d0db755ec5 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:39.548601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5649be9-9df5-4588-b199-09d0db755ec5 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:39.576384 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:39.581928 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796c35a-2a43-40b6-bc37-8d7a3f4d4747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:39.582521 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796c35a-2a43-40b6-bc37-8d7a3f4d4747 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:39.582802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796c35a-2a43-40b6-bc37-8d7a3f4d4747 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:39.583074 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796c35a-2a43-40b6-bc37-8d7a3f4d4747 None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:39.583764 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796c35a-2a43-40b6-bc37-8d7a3f4d4747 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:39.585178 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796c35a-2a43-40b6-bc37-8d7a3f4d4747 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:39.609593 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 431/860] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Oct 21 21:09:39 2022] POST /identity/v3/projects => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:09:41.799560 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce20596-bc12-43a8-ac2f-85b008cb985c None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:41.800060 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce20596-bc12-43a8-ac2f-85b008cb985c None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:41.800151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce20596-bc12-43a8-ac2f-85b008cb985c None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:41.801461 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 430/861] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:41.806833 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd665b-78d5-4382-b42a-000a96769dc7 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:41.807230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd665b-78d5-4382-b42a-000a96769dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:41.807389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd665b-78d5-4382-b42a-000a96769dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:41.826142 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-c1fd665b-78d5-4382-b42a-000a96769dc7 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:41.827313 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c1fd665b-78d5-4382-b42a-000a96769dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:41.891324 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 432/862] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:42.162137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce20596-bc12-43a8-ac2f-85b008cb985c None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:42.258538 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce20596-bc12-43a8-ac2f-85b008cb985c None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:42.290094 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af52452-41bf-4b2c-9184-f250eeacf946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:42.290716 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af52452-41bf-4b2c-9184-f250eeacf946 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:42.290993 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af52452-41bf-4b2c-9184-f250eeacf946 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:42.291261 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af52452-41bf-4b2c-9184-f250eeacf946 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:42.292389 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af52452-41bf-4b2c-9184-f250eeacf946 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:42.293831 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af52452-41bf-4b2c-9184-f250eeacf946 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:42.295362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 431/863] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:09:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:42.301446 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd665b-78d5-4382-b42a-000a96769dc7 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:42.395393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd665b-78d5-4382-b42a-000a96769dc7 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:42.409109 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:42.428724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919a262-3718-4f94-bdd8-7353bb5bdcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:42.429261 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919a262-3718-4f94-bdd8-7353bb5bdcda None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:42.429539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919a262-3718-4f94-bdd8-7353bb5bdcda None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:42.429811 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919a262-3718-4f94-bdd8-7353bb5bdcda None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:42.430459 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919a262-3718-4f94-bdd8-7353bb5bdcda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:42.431889 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919a262-3718-4f94-bdd8-7353bb5bdcda None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:42.453723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 433/864] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:09:42 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:09:44.585044 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8cb9e-5c66-495c-a3de-c4fc8de800fd None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:44.585601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8cb9e-5c66-495c-a3de-c4fc8de800fd None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:44.585601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8cb9e-5c66-495c-a3de-c4fc8de800fd None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:44.586898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 432/865] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:44.591973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291b9f7-a9b4-48f8-be63-5bb31d9bc8a4 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:44.592351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291b9f7-a9b4-48f8-be63-5bb31d9bc8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:44.592515 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291b9f7-a9b4-48f8-be63-5bb31d9bc8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:44.610673 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-6291b9f7-a9b4-48f8-be63-5bb31d9bc8a4 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:44.611873 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-6291b9f7-a9b4-48f8-be63-5bb31d9bc8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:44.675055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 434/866] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:44.943414 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8cb9e-5c66-495c-a3de-c4fc8de800fd None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:45.002406 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:45.040439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8cb9e-5c66-495c-a3de-c4fc8de800fd None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:45.071639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998166c-f5f9-4a7a-a8aa-8edd9cab0950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:45.072251 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998166c-f5f9-4a7a-a8aa-8edd9cab0950 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:45.072527 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998166c-f5f9-4a7a-a8aa-8edd9cab0950 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:45.072794 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998166c-f5f9-4a7a-a8aa-8edd9cab0950 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:45.082802 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998166c-f5f9-4a7a-a8aa-8edd9cab0950 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:09:45.087486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-6998166c-f5f9-4a7a-a8aa-8edd9cab0950 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:09:45.087486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:45.087486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:45.087486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:45.087486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:45.087486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:45.087486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:45.087486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:45.087486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:45.087486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:45.087486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:45.087486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:45.087486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:09:45.087486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:09:45.087486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:09:45.087486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:09:45.088561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:09:45.088561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:09:45.088561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:09:45.088561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:09:45.088561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:45.088561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:45.088561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:45.088561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:45.088561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:45.088561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:45.088561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:45.088561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:45.088561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:45.088561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:45.088561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:45.088561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:45.089451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:45.089451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:45.089451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:45.089451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:45.089451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:09:45.089451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:09:45.089451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:09:45.089451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:09:45.089451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:09:45.089451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:09:45.089451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:09:45.089451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:09:45.089451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 433/867] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:09:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:45.095082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291b9f7-a9b4-48f8-be63-5bb31d9bc8a4 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:45.185700 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:45.188804 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291b9f7-a9b4-48f8-be63-5bb31d9bc8a4 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:45.221788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f91d1-bec7-4190-8674-5c6c4c190065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:45.222375 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f91d1-bec7-4190-8674-5c6c4c190065 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:45.222646 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f91d1-bec7-4190-8674-5c6c4c190065 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:45.222911 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f91d1-bec7-4190-8674-5c6c4c190065 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:45.223555 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f91d1-bec7-4190-8674-5c6c4c190065 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:45.223881 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f91d1-bec7-4190-8674-5c6c4c190065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:45.225208 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f91d1-bec7-4190-8674-5c6c4c190065 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:45.232734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 435/868] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:09:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:45.238375 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998166c-f5f9-4a7a-a8aa-8edd9cab0950 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:45.332784 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998166c-f5f9-4a7a-a8aa-8edd9cab0950 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:45.354305 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:45.365708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06e295-733e-48c8-9dea-8354525bfa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:45.366344 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06e295-733e-48c8-9dea-8354525bfa12 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:45.366620 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06e295-733e-48c8-9dea-8354525bfa12 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:45.366887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06e295-733e-48c8-9dea-8354525bfa12 None admin] PATH_INFO: `/v3/users/6ef3b2be04a5453baaa4345060d62083` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:45.379851 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06e295-733e-48c8-9dea-8354525bfa12 None admin] RBAC: Authorizing `identity:get_user(user_id=6ef3b2be04a5453baaa4345060d62083)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:45.381281 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06e295-733e-48c8-9dea-8354525bfa12 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:45.382835 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 434/869] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:09:45 2022] GET /identity/v3/users/6ef3b2be04a5453baaa4345060d62083 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:45.388701 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f91d1-bec7-4190-8674-5c6c4c190065 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:45.484661 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f91d1-bec7-4190-8674-5c6c4c190065 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:45.517170 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d32381-3bb5-4c45-aee2-7680207da5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:45.517807 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d32381-3bb5-4c45-aee2-7680207da5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:45.518084 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d32381-3bb5-4c45-aee2-7680207da5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:45.518352 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d32381-3bb5-4c45-aee2-7680207da5e2 None admin] PATH_INFO: `/v3/projects/01997682191b4a6cb2c52664f60c907b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:45.519457 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d32381-3bb5-4c45-aee2-7680207da5e2 None admin] RBAC: Authorizing `identity:get_project(project_id=01997682191b4a6cb2c52664f60c907b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:45.520888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d32381-3bb5-4c45-aee2-7680207da5e2 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:45.522490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 436/870] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:09:45 2022] GET /identity/v3/projects/01997682191b4a6cb2c52664f60c907b => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:45.528338 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06e295-733e-48c8-9dea-8354525bfa12 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:45.622815 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06e295-733e-48c8-9dea-8354525bfa12 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:45.655699 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d4242-a7b2-493f-871b-dcda4a73aa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:45.656343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d4242-a7b2-493f-871b-dcda4a73aa59 None admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:45.656624 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d4242-a7b2-493f-871b-dcda4a73aa59 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:45.656898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d4242-a7b2-493f-871b-dcda4a73aa59 None admin] PATH_INFO: `/v3/projects/01997682191b4a6cb2c52664f60c907b/users/6ef3b2be04a5453baaa4345060d62083/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:45.658763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d4242-a7b2-493f-871b-dcda4a73aa59 None admin] RBAC: Authorizing `identity:create_grant(project_id=01997682191b4a6cb2c52664f60c907b, user_id=6ef3b2be04a5453baaa4345060d62083, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:45.660216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d4242-a7b2-493f-871b-dcda4a73aa59 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:45.671868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 435/871] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:09:45 2022] PUT /identity/v3/projects/01997682191b4a6cb2c52664f60c907b/users/6ef3b2be04a5453baaa4345060d62083/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:09:47.768175 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd32c5-d904-4880-b3df-dad41c9df7b4 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:47.768615 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd32c5-d904-4880-b3df-dad41c9df7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:47.768705 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd32c5-d904-4880-b3df-dad41c9df7b4 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:47.770044 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 437/872] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:47.775248 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfe9d3-628d-43f8-9671-bad73973740a None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:47.775639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfe9d3-628d-43f8-9671-bad73973740a None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:47.775820 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfe9d3-628d-43f8-9671-bad73973740a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:47.793356 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-f0bfe9d3-628d-43f8-9671-bad73973740a None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:47.794522 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f0bfe9d3-628d-43f8-9671-bad73973740a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:47.829648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:47.867385 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 436/873] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:48.143463 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd32c5-d904-4880-b3df-dad41c9df7b4 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:48.173524 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:48.240380 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd32c5-d904-4880-b3df-dad41c9df7b4 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:48.272133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca457ab-5d8d-4318-9b79-48ea4661aa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:48.272738 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca457ab-5d8d-4318-9b79-48ea4661aa8f None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:48.273014 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca457ab-5d8d-4318-9b79-48ea4661aa8f None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:48.273283 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca457ab-5d8d-4318-9b79-48ea4661aa8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:48.284100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca457ab-5d8d-4318-9b79-48ea4661aa8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:09:48.288810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-7ca457ab-5d8d-4318-9b79-48ea4661aa8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:09:48.288810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:09:48.288810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:09:48.288810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:09:48.288810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:09:48.288810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:09:48.288810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:09:48.288810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:09:48.288810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:09:48.288810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:09:48.288810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:09:48.288810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:09:48.288810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:09:48.288810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:09:48.288810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:09:48.288810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:09:48.289880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:09:48.289880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:09:48.289880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:09:48.289880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:09:48.289880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:09:48.289880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:09:48.289880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:09:48.289880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:09:48.289880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:09:48.289880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:09:48.289880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:09:48.289880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:09:48.289880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:09:48.289880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:09:48.289880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:09:48.289880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:09:48.291119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:09:48.291119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:09:48.291119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:09:48.291119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:09:48.291119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:09:48.291119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:09:48.291119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:09:48.291119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:09:48.291119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:09:48.291119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:09:48.291119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:09:48.291119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:09:48.291119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 438/874] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:09:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:09:48.296530 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfe9d3-628d-43f8-9671-bad73973740a None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:48.390436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfe9d3-628d-43f8-9671-bad73973740a None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:48.423151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c85764-82cd-47f9-9cd4-b91215868cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:48.423725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c85764-82cd-47f9-9cd4-b91215868cea None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:48.424003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c85764-82cd-47f9-9cd4-b91215868cea None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:48.424273 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c85764-82cd-47f9-9cd4-b91215868cea None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:48.424920 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c85764-82cd-47f9-9cd4-b91215868cea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:48.425222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c85764-82cd-47f9-9cd4-b91215868cea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:48.426596 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c85764-82cd-47f9-9cd4-b91215868cea None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:48.434257 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 437/875] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:09:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:48.440170 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca457ab-5d8d-4318-9b79-48ea4661aa8f None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:48.526114 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:48.536153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca457ab-5d8d-4318-9b79-48ea4661aa8f None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:48.567804 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a889e2d-d9ee-4e13-9d8d-8ec7f4a8927a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:48.568421 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a889e2d-d9ee-4e13-9d8d-8ec7f4a8927a None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:48.568699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a889e2d-d9ee-4e13-9d8d-8ec7f4a8927a None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:48.568968 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a889e2d-d9ee-4e13-9d8d-8ec7f4a8927a None admin] PATH_INFO: `/v3/users/6ef3b2be04a5453baaa4345060d62083` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:48.570072 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a889e2d-d9ee-4e13-9d8d-8ec7f4a8927a None admin] RBAC: Authorizing `identity:get_user(user_id=6ef3b2be04a5453baaa4345060d62083)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:48.571482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a889e2d-d9ee-4e13-9d8d-8ec7f4a8927a None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:48.573054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 439/876] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:09:48 2022] GET /identity/v3/users/6ef3b2be04a5453baaa4345060d62083 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:48.578695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c85764-82cd-47f9-9cd4-b91215868cea None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:48.609431 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:48.673909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c85764-82cd-47f9-9cd4-b91215868cea None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:48.706007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d81862-bf6d-4e45-8199-b380bf76e6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:48.706602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d81862-bf6d-4e45-8199-b380bf76e6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:48.706881 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d81862-bf6d-4e45-8199-b380bf76e6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:48.707154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d81862-bf6d-4e45-8199-b380bf76e6b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:48.771488 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d81862-bf6d-4e45-8199-b380bf76e6b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:48.772957 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d81862-bf6d-4e45-8199-b380bf76e6b2 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:48.800822 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:48.865811 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 438/877] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:09:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 21:09:48.871649 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a889e2d-d9ee-4e13-9d8d-8ec7f4a8927a None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:48.967557 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a889e2d-d9ee-4e13-9d8d-8ec7f4a8927a None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:48.999710 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb80bc7-9ec5-4a84-ab21-4a43eb4c54d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:49.000314 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb80bc7-9ec5-4a84-ab21-4a43eb4c54d6 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:49.000589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb80bc7-9ec5-4a84-ab21-4a43eb4c54d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:49.000857 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb80bc7-9ec5-4a84-ab21-4a43eb4c54d6 None admin] PATH_INFO: `/v3/projects/01997682191b4a6cb2c52664f60c907b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:49.001949 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb80bc7-9ec5-4a84-ab21-4a43eb4c54d6 None admin] RBAC: Authorizing `identity:get_project(project_id=01997682191b4a6cb2c52664f60c907b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:49.003344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb80bc7-9ec5-4a84-ab21-4a43eb4c54d6 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:49.004947 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 440/878] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:09:48 2022] GET /identity/v3/projects/01997682191b4a6cb2c52664f60c907b => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:49.011177 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d81862-bf6d-4e45-8199-b380bf76e6b2 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:49.105540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d81862-bf6d-4e45-8199-b380bf76e6b2 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:49.133144 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:49.138479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8189c6-412a-426b-a4dc-e463686f9ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:49.139066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8189c6-412a-426b-a4dc-e463686f9ffe None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:49.139343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8189c6-412a-426b-a4dc-e463686f9ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:49.139636 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8189c6-412a-426b-a4dc-e463686f9ffe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:49.140349 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8189c6-412a-426b-a4dc-e463686f9ffe None admin] RBAC: Adding query filter params (role.id=c514b065b7604094b899c586f79b0c5b, scope.project.id=01997682191b4a6cb2c52664f60c907b, user.id=6ef3b2be04a5453baaa4345060d62083) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:09:49.140650 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8189c6-412a-426b-a4dc-e463686f9ffe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:49.142074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8189c6-412a-426b-a4dc-e463686f9ffe None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:49.147529 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 439/879] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:09:49 2022] GET /identity/v3/role_assignments?user.id=6ef3b2be04a5453baaa4345060d62083&scope.project.id=01997682191b4a6cb2c52664f60c907b&role.id=c514b065b7604094b899c586f79b0c5b => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:51.263148 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bacd7-d3f0-4eb8-b6dd-efec610732be None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:51.263703 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bacd7-d3f0-4eb8-b6dd-efec610732be None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:51.263703 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bacd7-d3f0-4eb8-b6dd-efec610732be None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:51.265578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 441/880] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:51.270704 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9634e3-4969-4853-8c9f-bef8052966df None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:51.271116 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9634e3-4969-4853-8c9f-bef8052966df None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:51.271243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9634e3-4969-4853-8c9f-bef8052966df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:51.289725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-1b9634e3-4969-4853-8c9f-bef8052966df None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:51.290906 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-1b9634e3-4969-4853-8c9f-bef8052966df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:51.354697 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 440/881] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:51.626735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91bacd7-d3f0-4eb8-b6dd-efec610732be None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:51.653214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:51.724453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91bacd7-d3f0-4eb8-b6dd-efec610732be None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:51.757275 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373f405-83e5-498d-abe9-71efd7f134f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:51.757879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373f405-83e5-498d-abe9-71efd7f134f7 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:51.758158 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373f405-83e5-498d-abe9-71efd7f134f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:51.758428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373f405-83e5-498d-abe9-71efd7f134f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:51.759098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373f405-83e5-498d-abe9-71efd7f134f7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:51.760478 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373f405-83e5-498d-abe9-71efd7f134f7 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:51.776301 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 442/882] 10.222.0.13 () {60 vars in 1138 bytes} [Fri Oct 21 21:09:51 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:09:53.908787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f41ea3-8afb-41f1-a0dc-8432df04fc39 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:53.909327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f41ea3-8afb-41f1-a0dc-8432df04fc39 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:53.909327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f41ea3-8afb-41f1-a0dc-8432df04fc39 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:53.910675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 441/883] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:53.915881 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38173ec9-52df-4bf0-bdc4-18a512bf9365 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:53.916246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38173ec9-52df-4bf0-bdc4-18a512bf9365 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:53.916421 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38173ec9-52df-4bf0-bdc4-18a512bf9365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:53.934579 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-38173ec9-52df-4bf0-bdc4-18a512bf9365 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:53.935774 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-38173ec9-52df-4bf0-bdc4-18a512bf9365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:53.998703 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 443/884] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:54.267129 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f41ea3-8afb-41f1-a0dc-8432df04fc39 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:54.362372 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f41ea3-8afb-41f1-a0dc-8432df04fc39 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:54.387657 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:54.396542 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39568daf-ebeb-4a65-8656-08fd27ada8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:54.397154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39568daf-ebeb-4a65-8656-08fd27ada8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:54.397429 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39568daf-ebeb-4a65-8656-08fd27ada8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:54.397695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39568daf-ebeb-4a65-8656-08fd27ada8a7 None admin] PATH_INFO: `/v3/domains/79d2207b516f4a42b0131bdb9816ea81` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:54.407761 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39568daf-ebeb-4a65-8656-08fd27ada8a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=79d2207b516f4a42b0131bdb9816ea81)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:54.409204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39568daf-ebeb-4a65-8656-08fd27ada8a7 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:54.410749 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 442/885] 10.222.0.13 () {56 vars in 1183 bytes} [Fri Oct 21 21:09:54 2022] GET /identity/v3/domains/79d2207b516f4a42b0131bdb9816ea81 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:54.416711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38173ec9-52df-4bf0-bdc4-18a512bf9365 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:54.433258 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:54.511203 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38173ec9-52df-4bf0-bdc4-18a512bf9365 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:54.543750 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99e053-a09d-4951-a8e0-5cde4bebfea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:54.544291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99e053-a09d-4951-a8e0-5cde4bebfea9 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:54.544573 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99e053-a09d-4951-a8e0-5cde4bebfea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:54.544845 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99e053-a09d-4951-a8e0-5cde4bebfea9 None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:54.545496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99e053-a09d-4951-a8e0-5cde4bebfea9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:54.546922 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99e053-a09d-4951-a8e0-5cde4bebfea9 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:54.572507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 444/886] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Oct 21 21:09:54 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:09:56.719224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be21369-30c2-4a23-a700-df92b5c914ae None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:56.719782 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be21369-30c2-4a23-a700-df92b5c914ae None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:56.719782 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be21369-30c2-4a23-a700-df92b5c914ae None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:56.721115 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 443/887] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:56.726506 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba754c1-f545-472a-a5ae-f27a0934ff81 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:56.726908 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba754c1-f545-472a-a5ae-f27a0934ff81 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:56.727104 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba754c1-f545-472a-a5ae-f27a0934ff81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:56.746226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-fba754c1-f545-472a-a5ae-f27a0934ff81 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:56.747755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-fba754c1-f545-472a-a5ae-f27a0934ff81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:56.808331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:56.815007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 445/888] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:57.079935 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be21369-30c2-4a23-a700-df92b5c914ae None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:57.175488 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be21369-30c2-4a23-a700-df92b5c914ae None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:57.206640 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea948265-f42f-4519-a9ed-936981578565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:57.207256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea948265-f42f-4519-a9ed-936981578565 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:57.207531 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea948265-f42f-4519-a9ed-936981578565 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:57.207828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea948265-f42f-4519-a9ed-936981578565 None admin] PATH_INFO: `/v3/domains/79d2207b516f4a42b0131bdb9816ea81` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:57.208916 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea948265-f42f-4519-a9ed-936981578565 None admin] RBAC: Authorizing `identity:get_domain(domain_id=79d2207b516f4a42b0131bdb9816ea81)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:57.210337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea948265-f42f-4519-a9ed-936981578565 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:57.211913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 444/889] 10.222.0.13 () {56 vars in 1183 bytes} [Fri Oct 21 21:09:57 2022] GET /identity/v3/domains/79d2207b516f4a42b0131bdb9816ea81 => generated 267 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:09:57.218100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba754c1-f545-472a-a5ae-f27a0934ff81 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:57.311072 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba754c1-f545-472a-a5ae-f27a0934ff81 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:09:57.343117 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca91e8-5ea1-4866-8558-fbabe9fa30fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:09:57.343813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca91e8-5ea1-4866-8558-fbabe9fa30fd None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:57.344098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca91e8-5ea1-4866-8558-fbabe9fa30fd None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:57.344371 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca91e8-5ea1-4866-8558-fbabe9fa30fd None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:57.345023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca91e8-5ea1-4866-8558-fbabe9fa30fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:09:57.346427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca91e8-5ea1-4866-8558-fbabe9fa30fd None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:09:57.367905 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 446/890] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:09:57 2022] POST /identity/v3/users => generated 311 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:09:59.508967 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a472be-ecc8-46fd-b1c8-14ecfbeccd4e None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:59.509546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a472be-ecc8-46fd-b1c8-14ecfbeccd4e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:59.509546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a472be-ecc8-46fd-b1c8-14ecfbeccd4e None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:59.510886 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 445/891] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:09:59.516815 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c51fb3-89b3-479f-833f-90966e64e92e None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:09:59.517206 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c51fb3-89b3-479f-833f-90966e64e92e None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:09:59.517369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c51fb3-89b3-479f-833f-90966e64e92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:09:59.535625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-79c51fb3-89b3-479f-833f-90966e64e92e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:09:59.536812 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-79c51fb3-89b3-479f-833f-90966e64e92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:09:59.583304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:59.600929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 447/892] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:09:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:09:59.869556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a472be-ecc8-46fd-b1c8-14ecfbeccd4e None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:09:59.925267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:09:59.969002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a472be-ecc8-46fd-b1c8-14ecfbeccd4e None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:10:00.000350 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc10bb-d060-4bb0-bb29-c6f10e4912ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:10:00.000958 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc10bb-d060-4bb0-bb29-c6f10e4912ec None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:10:00.001231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc10bb-d060-4bb0-bb29-c6f10e4912ec None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:10:00.001499 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc10bb-d060-4bb0-bb29-c6f10e4912ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:10:00.011618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc10bb-d060-4bb0-bb29-c6f10e4912ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:10:00.016769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-bafc10bb-d060-4bb0-bb29-c6f10e4912ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:10:00.016769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:10:00.016769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:10:00.016769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:10:00.016769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:10:00.016769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:10:00.016769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:10:00.016769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:10:00.016769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:10:00.016769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:10:00.016769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:10:00.016769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:10:00.016769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:10:00.016769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:10:00.016769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:10:00.016769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:10:00.017922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:10:00.017922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:10:00.017922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:10:00.017922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:10:00.017922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:10:00.017922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:10:00.017922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:10:00.017922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:10:00.017922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:10:00.017922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:10:00.017922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:10:00.017922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:10:00.017922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:10:00.017922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:10:00.017922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:10:00.017922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:10:00.019012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:10:00.019012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:10:00.019012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:10:00.019012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:10:00.019012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:10:00.019012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:10:00.019012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:10:00.019012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:10:00.019012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:10:00.019012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:10:00.019012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:10:00.019012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:10:00.019012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 446/893] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:09:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:10:00.023769 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c51fb3-89b3-479f-833f-90966e64e92e None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:10:00.116358 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c51fb3-89b3-479f-833f-90966e64e92e None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:10:00.148469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2ada7-1f48-499b-b917-9a2fcca2dc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:10:00.148998 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2ada7-1f48-499b-b917-9a2fcca2dc20 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:10:00.149275 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2ada7-1f48-499b-b917-9a2fcca2dc20 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:10:00.149544 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2ada7-1f48-499b-b917-9a2fcca2dc20 None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:10:00.150187 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2ada7-1f48-499b-b917-9a2fcca2dc20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:10:00.150486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2ada7-1f48-499b-b917-9a2fcca2dc20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:10:00.151850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2ada7-1f48-499b-b917-9a2fcca2dc20 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:10:00.159428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 448/894] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:10:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:10:00.164856 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc10bb-d060-4bb0-bb29-c6f10e4912ec None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:10:00.258927 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc10bb-d060-4bb0-bb29-c6f10e4912ec None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:10:00.271954 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:10:00.291178 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ee3f7-d097-4246-a6ad-b66d86b41c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:10:00.291810 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ee3f7-d097-4246-a6ad-b66d86b41c99 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:10:00.292084 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ee3f7-d097-4246-a6ad-b66d86b41c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:10:00.292350 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ee3f7-d097-4246-a6ad-b66d86b41c99 None admin] PATH_INFO: `/v3/users/f57d9604e5ab4af8b2874d195506b69d` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:10:00.305096 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ee3f7-d097-4246-a6ad-b66d86b41c99 None admin] RBAC: Authorizing `identity:get_user(user_id=f57d9604e5ab4af8b2874d195506b69d)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:10:00.306517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ee3f7-d097-4246-a6ad-b66d86b41c99 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:10:00.308101 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 447/895] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:10:00 2022] GET /identity/v3/users/f57d9604e5ab4af8b2874d195506b69d => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:10:00.313779 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2ada7-1f48-499b-b917-9a2fcca2dc20 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:10:00.358552 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:10:00.409840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2ada7-1f48-499b-b917-9a2fcca2dc20 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:10:00.441399 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b2b79-65c4-40bc-9cea-baea2bb63704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:10:00.442012 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b2b79-65c4-40bc-9cea-baea2bb63704 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:10:00.442293 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b2b79-65c4-40bc-9cea-baea2bb63704 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:10:00.442567 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b2b79-65c4-40bc-9cea-baea2bb63704 None admin] PATH_INFO: `/v3/projects/39de56ea9eed403087cbff4b1e37a7c5` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:10:00.443705 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b2b79-65c4-40bc-9cea-baea2bb63704 None admin] RBAC: Authorizing `identity:get_project(project_id=39de56ea9eed403087cbff4b1e37a7c5)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:10:00.445121 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b2b79-65c4-40bc-9cea-baea2bb63704 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:10:00.446739 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 449/896] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:10:00 2022] GET /identity/v3/projects/39de56ea9eed403087cbff4b1e37a7c5 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:10:00.453054 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ee3f7-d097-4246-a6ad-b66d86b41c99 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:10:00.546768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ee3f7-d097-4246-a6ad-b66d86b41c99 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:10:00.579638 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72277c55-6ef6-43cd-bc4d-4b93c263463b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:10:00.580276 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72277c55-6ef6-43cd-bc4d-4b93c263463b None admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:10:00.580552 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72277c55-6ef6-43cd-bc4d-4b93c263463b None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:10:00.580819 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72277c55-6ef6-43cd-bc4d-4b93c263463b None admin] PATH_INFO: `/v3/projects/39de56ea9eed403087cbff4b1e37a7c5/users/f57d9604e5ab4af8b2874d195506b69d/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:10:00.582664 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72277c55-6ef6-43cd-bc4d-4b93c263463b None admin] RBAC: Authorizing `identity:create_grant(project_id=39de56ea9eed403087cbff4b1e37a7c5, user_id=f57d9604e5ab4af8b2874d195506b69d, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:10:00.584112 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72277c55-6ef6-43cd-bc4d-4b93c263463b None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:10:00.596436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 448/897] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:10:00 2022] PUT /identity/v3/projects/39de56ea9eed403087cbff4b1e37a7c5/users/f57d9604e5ab4af8b2874d195506b69d/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:10:02.693256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95926e-fff0-4913-b76a-b0f8c9bf69dc None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:10:02.693807 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95926e-fff0-4913-b76a-b0f8c9bf69dc None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:10:02.693807 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95926e-fff0-4913-b76a-b0f8c9bf69dc None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:10:02.695127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 450/898] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:10:02.700473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca83c1f-8a0f-419b-b0a6-5a437e847d21 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:10:02.700848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca83c1f-8a0f-419b-b0a6-5a437e847d21 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:10:02.701023 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca83c1f-8a0f-419b-b0a6-5a437e847d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:10:02.719029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-0ca83c1f-8a0f-419b-b0a6-5a437e847d21 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:10:02.720234 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0ca83c1f-8a0f-419b-b0a6-5a437e847d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:10:02.745327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:10:02.786743 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 449/899] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:10:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:10:03.054685 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95926e-fff0-4913-b76a-b0f8c9bf69dc None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:10:03.152116 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95926e-fff0-4913-b76a-b0f8c9bf69dc None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:10:03.165323 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:10:03.184318 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22fd80-4931-4a12-b752-a74c18bb2ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:10:03.184925 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22fd80-4931-4a12-b752-a74c18bb2ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:10:03.185200 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22fd80-4931-4a12-b752-a74c18bb2ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:10:03.185465 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22fd80-4931-4a12-b752-a74c18bb2ba6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:10:03.195293 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a22fd80-4931-4a12-b752-a74c18bb2ba6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:10:03.200021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-6a22fd80-4931-4a12-b752-a74c18bb2ba6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:10:03.200021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:10:03.200021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:10:03.200021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:10:03.200021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:10:03.200021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:10:03.200021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:10:03.200021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:10:03.200021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:10:03.200021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:10:03.200021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:10:03.200021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:10:03.200021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:10:03.200021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:10:03.200021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:10:03.200021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:10:03.201625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:10:03.201625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:10:03.201625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:10:03.201625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:10:03.201625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:10:03.201625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:10:03.201625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:10:03.201625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:10:03.201625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:10:03.201625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:10:03.201625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:10:03.201625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:10:03.201625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:10:03.201625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:10:03.201625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:10:03.201625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:10:03.202632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:10:03.202632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:10:03.202632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:10:03.202632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:10:03.202632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:10:03.202632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:10:03.202632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:10:03.202632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:10:03.202632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:10:03.202632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:10:03.202632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:10:03.202632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:10:03.202632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 451/900] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:10:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:10:03.207643 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca83c1f-8a0f-419b-b0a6-5a437e847d21 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:10:03.414326 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca83c1f-8a0f-419b-b0a6-5a437e847d21 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:10:03.446289 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbed428-066b-4d1e-94f3-3a3744b13090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:10:03.446877 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbed428-066b-4d1e-94f3-3a3744b13090 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:10:03.447153 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbed428-066b-4d1e-94f3-3a3744b13090 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:10:03.447422 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbed428-066b-4d1e-94f3-3a3744b13090 None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:10:03.448096 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbed428-066b-4d1e-94f3-3a3744b13090 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:10:03.448396 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbed428-066b-4d1e-94f3-3a3744b13090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:10:03.449739 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbed428-066b-4d1e-94f3-3a3744b13090 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:10:03.457402 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 450/901] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:10:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:10:03.463488 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22fd80-4931-4a12-b752-a74c18bb2ba6 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:10:03.558628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22fd80-4931-4a12-b752-a74c18bb2ba6 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:10:03.592355 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ddd56-cdc6-4b05-9eeb-fedb9a76ea41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:10:03.592963 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ddd56-cdc6-4b05-9eeb-fedb9a76ea41 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:10:03.593238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ddd56-cdc6-4b05-9eeb-fedb9a76ea41 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:10:03.593501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ddd56-cdc6-4b05-9eeb-fedb9a76ea41 None admin] PATH_INFO: `/v3/users/f57d9604e5ab4af8b2874d195506b69d` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:10:03.594599 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ddd56-cdc6-4b05-9eeb-fedb9a76ea41 None admin] RBAC: Authorizing `identity:get_user(user_id=f57d9604e5ab4af8b2874d195506b69d)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:10:03.596021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ddd56-cdc6-4b05-9eeb-fedb9a76ea41 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:10:03.597576 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 452/902] 10.222.0.13 () {56 vars in 1177 bytes} [Fri Oct 21 21:10:03 2022] GET /identity/v3/users/f57d9604e5ab4af8b2874d195506b69d => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:10:03.603249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbed428-066b-4d1e-94f3-3a3744b13090 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:10:03.627757 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:10:03.697934 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbed428-066b-4d1e-94f3-3a3744b13090 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:10:03.729829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbdeb07-dffe-4e12-8eed-20618806133d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:10:03.730427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdeb07-dffe-4e12-8eed-20618806133d None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:10:03.730703 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdeb07-dffe-4e12-8eed-20618806133d None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:10:03.731001 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdeb07-dffe-4e12-8eed-20618806133d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:10:03.795549 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbdeb07-dffe-4e12-8eed-20618806133d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:10:03.797015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbdeb07-dffe-4e12-8eed-20618806133d None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:10:03.821952 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:10:03.892218 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 451/903] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:10:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 21:10:03.898081 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ddd56-cdc6-4b05-9eeb-fedb9a76ea41 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:10:03.939481 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:10:03.992704 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ddd56-cdc6-4b05-9eeb-fedb9a76ea41 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:10:04.024230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d15e6c-77cf-4b01-9945-9103eec13543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:10:04.024838 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d15e6c-77cf-4b01-9945-9103eec13543 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:10:04.025114 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d15e6c-77cf-4b01-9945-9103eec13543 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:10:04.025380 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d15e6c-77cf-4b01-9945-9103eec13543 None admin] PATH_INFO: `/v3/projects/39de56ea9eed403087cbff4b1e37a7c5` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:10:04.026473 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d15e6c-77cf-4b01-9945-9103eec13543 None admin] RBAC: Authorizing `identity:get_project(project_id=39de56ea9eed403087cbff4b1e37a7c5)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:10:04.027889 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d15e6c-77cf-4b01-9945-9103eec13543 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:10:04.029539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 453/904] 10.222.0.13 () {56 vars in 1186 bytes} [Fri Oct 21 21:10:03 2022] GET /identity/v3/projects/39de56ea9eed403087cbff4b1e37a7c5 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:10:04.035693 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbdeb07-dffe-4e12-8eed-20618806133d None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:10:04.128758 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbdeb07-dffe-4e12-8eed-20618806133d None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:10:04.150471 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:10:04.161793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3158328-7bdc-45ef-ab14-2ee0fb44e5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:10:04.162376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3158328-7bdc-45ef-ab14-2ee0fb44e5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:10:04.162647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3158328-7bdc-45ef-ab14-2ee0fb44e5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:10:04.162913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3158328-7bdc-45ef-ab14-2ee0fb44e5c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:10:04.163625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3158328-7bdc-45ef-ab14-2ee0fb44e5c6 None admin] RBAC: Adding query filter params (role.id=c514b065b7604094b899c586f79b0c5b, scope.project.id=39de56ea9eed403087cbff4b1e37a7c5, user.id=f57d9604e5ab4af8b2874d195506b69d) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:10:04.163928 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3158328-7bdc-45ef-ab14-2ee0fb44e5c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:10:04.165339 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3158328-7bdc-45ef-ab14-2ee0fb44e5c6 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:10:04.170837 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 452/905] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:10:04 2022] GET /identity/v3/role_assignments?user.id=f57d9604e5ab4af8b2874d195506b69d&scope.project.id=39de56ea9eed403087cbff4b1e37a7c5&role.id=c514b065b7604094b899c586f79b0c5b => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:10:14.020448 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=45 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:10:14.168295 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=6 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:10:59.211614 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649d516-7c39-4458-88cb-f356ce70074c None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:10:59.211614 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649d516-7c39-4458-88cb-f356ce70074c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:10:59.211614 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649d516-7c39-4458-88cb-f356ce70074c None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:10:59.212377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 454/906] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:10:59.218566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b8726-ac6b-4afe-86b8-3d2be23fd71d None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:10:59.219120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b8726-ac6b-4afe-86b8-3d2be23fd71d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:10:59.219120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b8726-ac6b-4afe-86b8-3d2be23fd71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:10:59.240199 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-656b8726-ac6b-4afe-86b8-3d2be23fd71d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:10:59.241592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-656b8726-ac6b-4afe-86b8-3d2be23fd71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:10:59.327833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 453/907] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:10:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 21:10:59.642806 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649d516-7c39-4458-88cb-f356ce70074c None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:10:59.774601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649d516-7c39-4458-88cb-f356ce70074c None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:10:59.829373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c45532-3dc1-4830-9d87-c8e734ffa974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:10:59.830379 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c45532-3dc1-4830-9d87-c8e734ffa974 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:10:59.830820 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c45532-3dc1-4830-9d87-c8e734ffa974 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:10:59.831281 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c45532-3dc1-4830-9d87-c8e734ffa974 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:10:59.844705 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c45532-3dc1-4830-9d87-c8e734ffa974 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:10:59.846578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c45532-3dc1-4830-9d87-c8e734ffa974 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:10:59.863470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-60c45532-3dc1-4830-9d87-c8e734ffa974 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:10:59.863470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:10:59.863470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:10:59.863470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:10:59.863470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:10:59.863470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:10:59.863470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:10:59.863470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:10:59.863470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:10:59.863470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:10:59.863470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:10:59.863470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:10:59.863470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:10:59.863470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:10:59.863470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:10:59.863470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:10:59.865001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:10:59.865001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:10:59.865001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:10:59.865001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:10:59.865001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:10:59.865001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:10:59.865001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:10:59.865001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:10:59.865001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:10:59.865001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:10:59.865001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:10:59.865001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:10:59.865001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:10:59.865001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:10:59.865001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:10:59.865001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:10:59.865001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:10:59.865973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:10:59.865973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:10:59.865973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:10:59.865973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 455/908] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:10:59.873057 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b8726-ac6b-4afe-86b8-3d2be23fd71d None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:00.024097 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b8726-ac6b-4afe-86b8-3d2be23fd71d None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:00.056762 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:00.064048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9c244-4ee2-4c86-9668-4e230c1ca8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:00.064789 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9c244-4ee2-4c86-9668-4e230c1ca8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:00.065077 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9c244-4ee2-4c86-9668-4e230c1ca8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:00.065345 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9c244-4ee2-4c86-9668-4e230c1ca8b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:00.066006 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9c244-4ee2-4c86-9668-4e230c1ca8b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:00.066281 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9c244-4ee2-4c86-9668-4e230c1ca8b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:00.067802 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9c244-4ee2-4c86-9668-4e230c1ca8b7 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:00.082582 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 454/909] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:00.086777 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c45532-3dc1-4830-9d87-c8e734ffa974 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:00.151058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:00.220777 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c45532-3dc1-4830-9d87-c8e734ffa974 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:00.266484 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184ef92-c700-4f78-9ecf-bb39a6de6e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:00.267254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184ef92-c700-4f78-9ecf-bb39a6de6e11 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:00.267516 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184ef92-c700-4f78-9ecf-bb39a6de6e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:00.267953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184ef92-c700-4f78-9ecf-bb39a6de6e11 None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:00.268617 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184ef92-c700-4f78-9ecf-bb39a6de6e11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:00.270209 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184ef92-c700-4f78-9ecf-bb39a6de6e11 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:00.300911 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 456/910] 10.222.0.13 () {60 vars in 1132 bytes} [Fri Oct 21 21:11:00 2022] POST /identity/v3/users => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:11:02.692750 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81a280-ffd7-4c7f-aa11-4c6e7d0db584 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:02.693326 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81a280-ffd7-4c7f-aa11-4c6e7d0db584 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:02.693326 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81a280-ffd7-4c7f-aa11-4c6e7d0db584 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:02.694630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 455/911] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:11:02.701605 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cfa810-b5ed-47fd-be20-2dc73b1f595e None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:02.702121 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cfa810-b5ed-47fd-be20-2dc73b1f595e None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:02.702305 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cfa810-b5ed-47fd-be20-2dc73b1f595e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:02.721915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-60cfa810-b5ed-47fd-be20-2dc73b1f595e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:11:02.725980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-60cfa810-b5ed-47fd-be20-2dc73b1f595e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:11:02.809926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 457/912] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:11:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:11:03.110306 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81a280-ffd7-4c7f-aa11-4c6e7d0db584 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:03.239642 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81a280-ffd7-4c7f-aa11-4c6e7d0db584 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:03.285174 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604f257-75d1-42b1-9f8e-e25b7381bdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:03.285884 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604f257-75d1-42b1-9f8e-e25b7381bdd8 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:03.286184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604f257-75d1-42b1-9f8e-e25b7381bdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:03.286505 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604f257-75d1-42b1-9f8e-e25b7381bdd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:03.299873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604f257-75d1-42b1-9f8e-e25b7381bdd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:11:03.305245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-3604f257-75d1-42b1-9f8e-e25b7381bdd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:11:03.305245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:03.305245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:03.305245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:03.305245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:03.305245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:03.305245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:03.305245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:03.305245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:03.305245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:03.305245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:03.305245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:03.305245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:11:03.305245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:11:03.305245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:11:03.305245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:11:03.306478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:11:03.306478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:11:03.306478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:11:03.306478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:11:03.306478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:03.306478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:03.306478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:03.306478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:03.306478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:03.306478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:03.306478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:03.306478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:03.306478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:03.306478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:03.306478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:03.306478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:03.307352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:03.307352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:03.307352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:03.307352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:03.307352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:11:03.307352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:11:03.307352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:11:03.307352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:11:03.307352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:11:03.307352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:11:03.307352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:11:03.307352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:03.307352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 456/913] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Oct 21 21:11:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:03.312887 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cfa810-b5ed-47fd-be20-2dc73b1f595e None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:03.357400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:03.422264 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cfa810-b5ed-47fd-be20-2dc73b1f595e None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:03.458410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd5700-b6e2-4348-a362-eaa9e99b854f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:03.459083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd5700-b6e2-4348-a362-eaa9e99b854f None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:03.459365 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd5700-b6e2-4348-a362-eaa9e99b854f None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:03.459673 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd5700-b6e2-4348-a362-eaa9e99b854f None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:03.460419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd5700-b6e2-4348-a362-eaa9e99b854f None admin] RBAC: Adding query filter params (name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:03.460797 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd5700-b6e2-4348-a362-eaa9e99b854f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:03.462315 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd5700-b6e2-4348-a362-eaa9e99b854f None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:03.472022 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 458/914] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Oct 21 21:11:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:03.478247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604f257-75d1-42b1-9f8e-e25b7381bdd8 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:03.532329 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:03.586919 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604f257-75d1-42b1-9f8e-e25b7381bdd8 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:03.621294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f6550-18e5-4fb6-8f85-6b7184a54554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:03.621955 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f6550-18e5-4fb6-8f85-6b7184a54554 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:03.622235 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f6550-18e5-4fb6-8f85-6b7184a54554 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:03.622508 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f6550-18e5-4fb6-8f85-6b7184a54554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:03.632955 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f6550-18e5-4fb6-8f85-6b7184a54554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:03.634704 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f6550-18e5-4fb6-8f85-6b7184a54554 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:03.649661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-eb8f6550-18e5-4fb6-8f85-6b7184a54554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:11:03.649661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:03.649661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:03.649661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:03.649661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:03.649661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:03.649661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:03.649661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:03.649661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:03.649661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:03.649661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:03.649661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:03.649661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:11:03.649661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:11:03.649661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:11:03.649661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:11:03.650778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:03.650778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:03.650778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:03.650778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:03.650778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:03.650778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:03.650778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:03.650778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:03.650778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:03.650778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:03.650778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:03.650778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:03.650778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:03.650778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:03.650778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:03.650778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:03.650778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:11:03.656473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:11:03.656473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:11:03.656473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:03.656473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 457/915] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:11:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:03.656720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd5700-b6e2-4348-a362-eaa9e99b854f None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:03.765783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:03.769059 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd5700-b6e2-4348-a362-eaa9e99b854f None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:03.807046 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50531b6e-5bb5-4b15-b44c-2284a57fc227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:03.807727 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50531b6e-5bb5-4b15-b44c-2284a57fc227 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:03.808012 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50531b6e-5bb5-4b15-b44c-2284a57fc227 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:03.808283 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50531b6e-5bb5-4b15-b44c-2284a57fc227 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:03.808949 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50531b6e-5bb5-4b15-b44c-2284a57fc227 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:03.809249 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50531b6e-5bb5-4b15-b44c-2284a57fc227 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:03.810610 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50531b6e-5bb5-4b15-b44c-2284a57fc227 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:03.823858 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 459/916] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:03.831837 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f6550-18e5-4fb6-8f85-6b7184a54554 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:03.948520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f6550-18e5-4fb6-8f85-6b7184a54554 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:03.950105 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:03.987296 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d7963-3a12-4d3b-971b-7b5a98f24437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:03.987973 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d7963-3a12-4d3b-971b-7b5a98f24437 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:03.988249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d7963-3a12-4d3b-971b-7b5a98f24437 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:03.988519 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d7963-3a12-4d3b-971b-7b5a98f24437 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:03.998788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d7963-3a12-4d3b-971b-7b5a98f24437 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:04.000365 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d7963-3a12-4d3b-971b-7b5a98f24437 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:04.012971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-226d7963-3a12-4d3b-971b-7b5a98f24437 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 21 21:11:04.012971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:04.012971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:04.012971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:04.012971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:04.012971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:04.012971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:04.012971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:04.012971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:04.012971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:04.012971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:04.012971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:04.012971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:11:04.012971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:11:04.012971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:11:04.012971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:04.014047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:04.015007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:04.015007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:04.015007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:11:04.015007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:11:04.015007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:11:04.015007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:11:04.015007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:11:04.015007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:11:04.015007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 21 21:11:04.015007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:04.015007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 458/917] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:11:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:04.020341 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50531b6e-5bb5-4b15-b44c-2284a57fc227 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:04.133784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50531b6e-5bb5-4b15-b44c-2284a57fc227 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:04.172152 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c61b7-e3d9-4767-8aa1-f389bfe3cde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:04.172806 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c61b7-e3d9-4767-8aa1-f389bfe3cde2 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:04.173128 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c61b7-e3d9-4767-8aa1-f389bfe3cde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:04.175013 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c61b7-e3d9-4767-8aa1-f389bfe3cde2 None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:04.175771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c61b7-e3d9-4767-8aa1-f389bfe3cde2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:04.176086 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c61b7-e3d9-4767-8aa1-f389bfe3cde2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:04.177517 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c61b7-e3d9-4767-8aa1-f389bfe3cde2 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:04.197103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 460/918] 10.222.0.13 () {56 vars in 1143 bytes} [Fri Oct 21 21:11:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:04.202866 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d7963-3a12-4d3b-971b-7b5a98f24437 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:04.319422 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d7963-3a12-4d3b-971b-7b5a98f24437 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:04.354013 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca9ee4-cf10-46ff-ad6a-c1bdbfc05c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:04.354636 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca9ee4-cf10-46ff-ad6a-c1bdbfc05c9f None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:04.354908 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca9ee4-cf10-46ff-ad6a-c1bdbfc05c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:04.355171 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca9ee4-cf10-46ff-ad6a-c1bdbfc05c9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:04.368045 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca9ee4-cf10-46ff-ad6a-c1bdbfc05c9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:04.370515 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca9ee4-cf10-46ff-ad6a-c1bdbfc05c9f None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:04.387543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-75ca9ee4-cf10-46ff-ad6a-c1bdbfc05c9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:11:04.387543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:04.387543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:04.387543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:04.387543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:04.387543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:04.387543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:04.387543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:04.387543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:04.387543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:04.387543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:04.387543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:04.387543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:11:04.387543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:11:04.387543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:11:04.387543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:11:04.388579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:04.388579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:04.388579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:04.388579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:04.388579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:04.388579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:04.388579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:04.388579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:04.388579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:04.388579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:04.388579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:04.388579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:04.388579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:04.388579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:04.388579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:04.388579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:04.388579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:11:04.390215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:11:04.390215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:11:04.390215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:04.390215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 459/919] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:04.394519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c61b7-e3d9-4767-8aa1-f389bfe3cde2 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:04.416084 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:04.507493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c61b7-e3d9-4767-8aa1-f389bfe3cde2 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:04.546067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203c17fd-df7a-4391-9e25-d02f8c5f2b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:04.549289 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c17fd-df7a-4391-9e25-d02f8c5f2b2b None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:04.549289 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c17fd-df7a-4391-9e25-d02f8c5f2b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:04.549289 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c17fd-df7a-4391-9e25-d02f8c5f2b2b None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:04.549289 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203c17fd-df7a-4391-9e25-d02f8c5f2b2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:04.549289 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203c17fd-df7a-4391-9e25-d02f8c5f2b2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:04.549726 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203c17fd-df7a-4391-9e25-d02f8c5f2b2b None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:04.564054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 461/920] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:11:04.570703 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca9ee4-cf10-46ff-ad6a-c1bdbfc05c9f None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:04.592761 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:04.696278 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca9ee4-cf10-46ff-ad6a-c1bdbfc05c9f None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:04.743764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976de14-c241-4f42-8bc1-d581af9928df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:04.744469 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976de14-c241-4f42-8bc1-d581af9928df None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:04.744746 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976de14-c241-4f42-8bc1-d581af9928df None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:04.745015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976de14-c241-4f42-8bc1-d581af9928df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:04.751897 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976de14-c241-4f42-8bc1-d581af9928df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:04.753550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976de14-c241-4f42-8bc1-d581af9928df None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:04.766617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-9976de14-c241-4f42-8bc1-d581af9928df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:11:04.766617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:04.766617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:04.766617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:04.766617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:04.766617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:04.766617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:04.766617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:04.766617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:04.766617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:04.766617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:04.766617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:04.766617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:11:04.766617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:11:04.766617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:11:04.767764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:11:04.767764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:04.767764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:04.767764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:04.767764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:04.767764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:04.767764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:04.767764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:04.767764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:04.767764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:04.767764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:04.767764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:04.767764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:04.767764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:04.767764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:04.767764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:04.767764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:04.768727 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:11:04.768727 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:11:04.768727 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:11:04.768727 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:11:04.768727 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:11:04.768727 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:11:04.768727 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:11:04.768727 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:04.768727 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 460/921] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:11:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:04.774222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203c17fd-df7a-4391-9e25-d02f8c5f2b2b None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:04.870757 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:04.897695 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203c17fd-df7a-4391-9e25-d02f8c5f2b2b None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:04.933731 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf6620-3a9d-4705-ab3b-2b4bdc6ecce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:04.934343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf6620-3a9d-4705-ab3b-2b4bdc6ecce5 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:04.934624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf6620-3a9d-4705-ab3b-2b4bdc6ecce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:04.934893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf6620-3a9d-4705-ab3b-2b4bdc6ecce5 None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:04.935593 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf6620-3a9d-4705-ab3b-2b4bdc6ecce5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:04.935908 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf6620-3a9d-4705-ab3b-2b4bdc6ecce5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:04.937324 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf6620-3a9d-4705-ab3b-2b4bdc6ecce5 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:04.955363 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 462/922] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:11:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:04.956853 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976de14-c241-4f42-8bc1-d581af9928df None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:05.044172 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:05.072559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976de14-c241-4f42-8bc1-d581af9928df None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:05.114204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef3891-4d71-4dc5-b637-08df5eaa5f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:05.115230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef3891-4d71-4dc5-b637-08df5eaa5f26 None admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:05.115641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef3891-4d71-4dc5-b637-08df5eaa5f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:05.115915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef3891-4d71-4dc5-b637-08df5eaa5f26 None admin] PATH_INFO: `/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/124f2b5b95e8450ea6867f6868b9e810/roles/b0f9a090148d4a0eb6456472edec3d5a` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:05.133645 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef3891-4d71-4dc5-b637-08df5eaa5f26 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b2c33250ef84d79a4a4742fb4252156, user_id=124f2b5b95e8450ea6867f6868b9e810, role_id=b0f9a090148d4a0eb6456472edec3d5a)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:05.135456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef3891-4d71-4dc5-b637-08df5eaa5f26 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:05.154030 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 461/923] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:11:04 2022] PUT /identity/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/124f2b5b95e8450ea6867f6868b9e810/roles/b0f9a090148d4a0eb6456472edec3d5a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:11:07.483054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd488bc-598d-49c0-9bc2-f4af7e30bbc9 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:07.483663 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd488bc-598d-49c0-9bc2-f4af7e30bbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:07.483663 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd488bc-598d-49c0-9bc2-f4af7e30bbc9 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:07.485102 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 463/924] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:11:07.490714 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711df23-7c62-48c5-9833-832d8fb1bcd5 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:07.491192 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711df23-7c62-48c5-9833-832d8fb1bcd5 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:07.491428 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711df23-7c62-48c5-9833-832d8fb1bcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:07.511768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-7711df23-7c62-48c5-9833-832d8fb1bcd5 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:11:07.513035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-7711df23-7c62-48c5-9833-832d8fb1bcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:11:07.581899 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 462/925] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:11:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:11:07.880028 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd488bc-598d-49c0-9bc2-f4af7e30bbc9 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:07.986741 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd488bc-598d-49c0-9bc2-f4af7e30bbc9 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:08.024205 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c2457-c396-4f92-9675-16b4ae4d77dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:08.024836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c2457-c396-4f92-9675-16b4ae4d77dd None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:08.025628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c2457-c396-4f92-9675-16b4ae4d77dd None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:08.025906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c2457-c396-4f92-9675-16b4ae4d77dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:08.032537 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:08.036518 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c2457-c396-4f92-9675-16b4ae4d77dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:11:08.041559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-0f3c2457-c396-4f92-9675-16b4ae4d77dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:11:08.041559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:08.041559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:08.041559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:08.041559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:08.041559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:08.041559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:08.041559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:08.041559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:08.041559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:08.041559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:08.041559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:08.041559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:11:08.041559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:11:08.041559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:11:08.041559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:11:08.042840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:11:08.042840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:11:08.042840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:11:08.042840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:11:08.042840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:08.042840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:08.042840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:08.042840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:08.042840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:08.042840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:08.042840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:08.042840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:08.042840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:08.042840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:08.042840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:08.042840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:08.043830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:08.043830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:08.043830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:08.043830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:08.043830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:11:08.043830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:11:08.043830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:11:08.043830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:11:08.043830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:11:08.043830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:11:08.043830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 21:11:08.043830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:11:08.043830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 464/926] 10.222.0.13 () {56 vars in 1102 bytes} [Fri Oct 21 21:11:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:08.048826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711df23-7c62-48c5-9833-832d8fb1bcd5 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:08.107745 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:08.154188 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711df23-7c62-48c5-9833-832d8fb1bcd5 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:08.189482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c77d7-7083-4466-a53e-da946e34060d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:08.190083 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c77d7-7083-4466-a53e-da946e34060d None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:08.190358 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c77d7-7083-4466-a53e-da946e34060d None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:08.190626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c77d7-7083-4466-a53e-da946e34060d None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:08.191272 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c77d7-7083-4466-a53e-da946e34060d None admin] RBAC: Adding query filter params (name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:08.191584 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c77d7-7083-4466-a53e-da946e34060d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:08.192999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c77d7-7083-4466-a53e-da946e34060d None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:08.201943 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 463/927] 10.222.0.13 () {56 vars in 1103 bytes} [Fri Oct 21 21:11:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:08.208855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c2457-c396-4f92-9675-16b4ae4d77dd None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:08.333169 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c2457-c396-4f92-9675-16b4ae4d77dd None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:08.370513 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d0d24-e300-4560-8ed6-5a5098487187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:08.371359 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d0d24-e300-4560-8ed6-5a5098487187 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:08.371670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d0d24-e300-4560-8ed6-5a5098487187 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:08.371940 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d0d24-e300-4560-8ed6-5a5098487187 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:08.383805 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d0d24-e300-4560-8ed6-5a5098487187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:08.386377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d0d24-e300-4560-8ed6-5a5098487187 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:08.401578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-482d0d24-e300-4560-8ed6-5a5098487187 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:11:08.401578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:08.401578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:08.401578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:08.401578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:08.401578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:08.401578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:08.401578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:08.401578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:08.401578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:08.401578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:08.401578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:08.401578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:11:08.401578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:11:08.401578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:11:08.401578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:11:08.402583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:08.402583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:08.402583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:08.402583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:08.402583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:08.402583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:08.402583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:08.402583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:08.402583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:08.402583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:08.402583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:08.402583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:08.402583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:08.402583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:08.402583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:08.402583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:08.402583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:11:08.403994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:11:08.403994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:11:08.403994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:11:08.403994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 465/928] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:11:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:08.408801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c77d7-7083-4466-a53e-da946e34060d None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:08.524418 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c77d7-7083-4466-a53e-da946e34060d None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:08.543066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:08.561253 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b20650-e20b-4d9e-9048-4bbd2e5b9cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:08.561948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b20650-e20b-4d9e-9048-4bbd2e5b9cde None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:08.562228 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b20650-e20b-4d9e-9048-4bbd2e5b9cde None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:08.562511 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b20650-e20b-4d9e-9048-4bbd2e5b9cde None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:08.563164 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b20650-e20b-4d9e-9048-4bbd2e5b9cde None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:08.563533 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b20650-e20b-4d9e-9048-4bbd2e5b9cde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:08.564931 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b20650-e20b-4d9e-9048-4bbd2e5b9cde None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:08.577475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 464/929] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:11:08.583144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d0d24-e300-4560-8ed6-5a5098487187 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:08.637192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:08.689133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d0d24-e300-4560-8ed6-5a5098487187 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:08.723556 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73526549-825c-4e89-a7cf-c53daea73e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:08.724212 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73526549-825c-4e89-a7cf-c53daea73e1d None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:08.724493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73526549-825c-4e89-a7cf-c53daea73e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:08.724764 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73526549-825c-4e89-a7cf-c53daea73e1d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:08.732509 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73526549-825c-4e89-a7cf-c53daea73e1d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:08.733949 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73526549-825c-4e89-a7cf-c53daea73e1d None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:08.746553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-73526549-825c-4e89-a7cf-c53daea73e1d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 21 21:11:08.746553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:08.746553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:08.746553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:08.746553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:08.746553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:08.746553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:08.746553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:08.746553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:08.746553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:08.746553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:08.746553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:08.746553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:11:08.746553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:11:08.746553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:11:08.746553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:08.747995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:08.749239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:08.749239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:08.749239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:11:08.749239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:11:08.749239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:11:08.749239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:11:08.749239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:11:08.749239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:11:08.749239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 21 21:11:08.749239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:11:08.749239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 466/930] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:11:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:11:08.754083 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b20650-e20b-4d9e-9048-4bbd2e5b9cde None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:08.865895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b20650-e20b-4d9e-9048-4bbd2e5b9cde None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:08.903171 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506837e-67a0-4813-abec-b1df32713e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:08.903752 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506837e-67a0-4813-abec-b1df32713e66 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:08.904035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506837e-67a0-4813-abec-b1df32713e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:08.904300 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506837e-67a0-4813-abec-b1df32713e66 None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:08.905018 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506837e-67a0-4813-abec-b1df32713e66 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:08.905322 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506837e-67a0-4813-abec-b1df32713e66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:08.906749 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506837e-67a0-4813-abec-b1df32713e66 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:08.924268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 465/931] 10.222.0.13 () {56 vars in 1143 bytes} [Fri Oct 21 21:11:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:08.930227 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73526549-825c-4e89-a7cf-c53daea73e1d None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:09.044766 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73526549-825c-4e89-a7cf-c53daea73e1d None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:09.055401 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:09.081929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921883e1-818c-47da-aaea-ba8259c8f2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:09.082540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921883e1-818c-47da-aaea-ba8259c8f2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:09.082821 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921883e1-818c-47da-aaea-ba8259c8f2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:09.083099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921883e1-818c-47da-aaea-ba8259c8f2f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:09.150832 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921883e1-818c-47da-aaea-ba8259c8f2f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:09.152303 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921883e1-818c-47da-aaea-ba8259c8f2f1 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:09.258579 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 467/932] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:11:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 21:11:09.264854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506837e-67a0-4813-abec-b1df32713e66 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:09.302014 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:09.372293 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506837e-67a0-4813-abec-b1df32713e66 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:09.408173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da624e-18bf-4edb-839c-461a3a62fbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:09.408816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da624e-18bf-4edb-839c-461a3a62fbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:09.409092 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da624e-18bf-4edb-839c-461a3a62fbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:09.409354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da624e-18bf-4edb-839c-461a3a62fbf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:09.419619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da624e-18bf-4edb-839c-461a3a62fbf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:09.421126 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da624e-18bf-4edb-839c-461a3a62fbf6 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:09.437173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-e8da624e-18bf-4edb-839c-461a3a62fbf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:11:09.437173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:09.437173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:09.437173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:09.437173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:09.437173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:09.437173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:09.437173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:09.437173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:09.437173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:09.437173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:09.437173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:09.437173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:11:09.437173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:11:09.437173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:11:09.437173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:11:09.438348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:09.438348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:09.438348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:09.438348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:09.438348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:09.438348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:09.438348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:09.438348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:09.438348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:09.438348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:09.438348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:09.438348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:09.438348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:09.438348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:09.438348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:09.438348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:09.438348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:11:09.439211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:11:09.439211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:11:09.439211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:09.439211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 466/933] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:11:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:09.444945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921883e1-818c-47da-aaea-ba8259c8f2f1 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:09.458100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:09.561245 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921883e1-818c-47da-aaea-ba8259c8f2f1 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:09.597445 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e2fd4-8fdf-4335-87ce-19106adfc925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:09.598094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e2fd4-8fdf-4335-87ce-19106adfc925 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:09.598319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e2fd4-8fdf-4335-87ce-19106adfc925 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:09.598587 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e2fd4-8fdf-4335-87ce-19106adfc925 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:09.599222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e2fd4-8fdf-4335-87ce-19106adfc925 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:09.599522 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e2fd4-8fdf-4335-87ce-19106adfc925 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:09.602343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e2fd4-8fdf-4335-87ce-19106adfc925 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:09.615709 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 468/934] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:09.621712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da624e-18bf-4edb-839c-461a3a62fbf6 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:09.708850 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:09.724830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da624e-18bf-4edb-839c-461a3a62fbf6 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:09.764036 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65687e-2e16-4dd5-9db6-7127cb7be617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:09.764669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65687e-2e16-4dd5-9db6-7127cb7be617 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:09.764952 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65687e-2e16-4dd5-9db6-7127cb7be617 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:09.765277 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65687e-2e16-4dd5-9db6-7127cb7be617 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:09.773341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65687e-2e16-4dd5-9db6-7127cb7be617 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:09.774886 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65687e-2e16-4dd5-9db6-7127cb7be617 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:09.788721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-7d65687e-2e16-4dd5-9db6-7127cb7be617 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:11:09.788721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:09.788721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:09.788721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:09.788721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:09.788721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:09.788721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:09.788721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:09.788721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:09.788721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:09.788721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:09.788721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:09.788721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:11:09.788721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:11:09.788721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:11:09.790816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:11:09.790816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:09.790816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:09.790816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:09.790816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:09.790816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:09.790816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:09.790816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:09.790816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:09.790816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:09.790816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:09.790816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:09.790816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:09.790816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:09.790816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:09.790816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:09.790816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:09.791948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:11:09.791948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:11:09.791948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:11:09.791948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:11:09.791948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:11:09.791948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:11:09.791948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:11:09.791948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:09.791948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 467/935] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:11:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 21:11:09.796549 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e2fd4-8fdf-4335-87ce-19106adfc925 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:09.872274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:09.906431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e2fd4-8fdf-4335-87ce-19106adfc925 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:09.942783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910c643-0ac3-4651-a87f-29b7a90eb1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:09.943467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910c643-0ac3-4651-a87f-29b7a90eb1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:09.943772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910c643-0ac3-4651-a87f-29b7a90eb1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:09.944045 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910c643-0ac3-4651-a87f-29b7a90eb1f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:09.944844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910c643-0ac3-4651-a87f-29b7a90eb1f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:09.945238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910c643-0ac3-4651-a87f-29b7a90eb1f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:09.946808 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910c643-0ac3-4651-a87f-29b7a90eb1f2 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:09.960816 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 469/936] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:11:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:09.967672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65687e-2e16-4dd5-9db6-7127cb7be617 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:10.079699 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65687e-2e16-4dd5-9db6-7127cb7be617 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:10.115229 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48ba72-a1f9-419e-9493-2dfc8a6701f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:10.116032 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48ba72-a1f9-419e-9493-2dfc8a6701f4 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:10.116426 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48ba72-a1f9-419e-9493-2dfc8a6701f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:10.116796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48ba72-a1f9-419e-9493-2dfc8a6701f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:10.117609 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48ba72-a1f9-419e-9493-2dfc8a6701f4 None admin] RBAC: Adding query filter params (role.id=b0f9a090148d4a0eb6456472edec3d5a, scope.project.id=8b2c33250ef84d79a4a4742fb4252156, user.id=124f2b5b95e8450ea6867f6868b9e810) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:10.118015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48ba72-a1f9-419e-9493-2dfc8a6701f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:10.120070 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48ba72-a1f9-419e-9493-2dfc8a6701f4 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:10.139686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 468/937] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:11:09 2022] GET /identity/v3/role_assignments?user.id=124f2b5b95e8450ea6867f6868b9e810&scope.project.id=8b2c33250ef84d79a4a4742fb4252156&role.id=b0f9a090148d4a0eb6456472edec3d5a => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:12.447591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71145a4-58e6-49dd-bddc-566e6c1a67e2 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:12.447591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71145a4-58e6-49dd-bddc-566e6c1a67e2 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:12.447591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71145a4-58e6-49dd-bddc-566e6c1a67e2 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:12.447591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 470/938] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:11:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:11:12.453437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f45ff-fa9b-413b-8b0c-e39cea714880 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:12.453902 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f45ff-fa9b-413b-8b0c-e39cea714880 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:12.454067 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f45ff-fa9b-413b-8b0c-e39cea714880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:12.475035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-054f45ff-fa9b-413b-8b0c-e39cea714880 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:11:12.476266 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-054f45ff-fa9b-413b-8b0c-e39cea714880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:11:12.479894 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:12.550028 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 469/939] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:11:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:11:12.843955 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71145a4-58e6-49dd-bddc-566e6c1a67e2 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:12.960755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71145a4-58e6-49dd-bddc-566e6c1a67e2 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:12.991308 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:12.997780 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e782e8-5f9b-4efc-a136-b1329955175f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:12.998403 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e782e8-5f9b-4efc-a136-b1329955175f None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:12.998677 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e782e8-5f9b-4efc-a136-b1329955175f None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:12.998961 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e782e8-5f9b-4efc-a136-b1329955175f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:13.012233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e782e8-5f9b-4efc-a136-b1329955175f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:11:13.017431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-c0e782e8-5f9b-4efc-a136-b1329955175f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:11:13.017431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:13.017431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:13.017431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:13.017431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:13.017431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:13.017431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:13.017431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:13.017431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:13.017431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:13.017431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:13.017431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:13.017431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:11:13.017431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:11:13.017431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:11:13.017431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:11:13.018393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:11:13.018393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:11:13.018393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:11:13.018393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:11:13.018393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:13.018393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:13.018393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:13.018393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:13.018393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:13.018393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:13.018393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:13.018393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:13.018393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:13.018393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:13.018393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:13.018393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:13.019500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:13.019500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:13.019500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:13.019500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:13.019500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:11:13.019500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:11:13.019500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:11:13.019500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:11:13.019500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:11:13.019500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:11:13.019500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:11:13.019500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:11:13.019500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 471/940] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:11:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:13.024695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f45ff-fa9b-413b-8b0c-e39cea714880 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:13.123677 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f45ff-fa9b-413b-8b0c-e39cea714880 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:13.157694 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd141abd-cb59-42fa-a2cb-12dd9bcede3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:13.158300 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd141abd-cb59-42fa-a2cb-12dd9bcede3b None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:13.158592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd141abd-cb59-42fa-a2cb-12dd9bcede3b None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:13.158841 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd141abd-cb59-42fa-a2cb-12dd9bcede3b None admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:13.159491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd141abd-cb59-42fa-a2cb-12dd9bcede3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:13.159819 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd141abd-cb59-42fa-a2cb-12dd9bcede3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:13.161179 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd141abd-cb59-42fa-a2cb-12dd9bcede3b None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:13.169871 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 470/941] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:11:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:11:13.175487 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e782e8-5f9b-4efc-a136-b1329955175f None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:13.287671 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e782e8-5f9b-4efc-a136-b1329955175f None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:13.326940 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82b6d9-3893-4908-8ff1-bfa6e5952881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:13.327572 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82b6d9-3893-4908-8ff1-bfa6e5952881 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:13.327881 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82b6d9-3893-4908-8ff1-bfa6e5952881 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:13.328151 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82b6d9-3893-4908-8ff1-bfa6e5952881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:13.339090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82b6d9-3893-4908-8ff1-bfa6e5952881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:13.340581 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82b6d9-3893-4908-8ff1-bfa6e5952881 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:13.357629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-5b82b6d9-3893-4908-8ff1-bfa6e5952881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:11:13.357629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:13.357629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:13.357629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:13.357629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:13.357629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:13.357629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:13.357629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:13.357629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:13.357629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:13.357629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:13.357629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:13.357629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:11:13.357629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:11:13.357629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:11:13.357629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:11:13.358850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:13.358850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:13.358850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:13.358850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:13.358850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:13.358850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:13.358850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:13.358850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:13.358850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:13.358850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:13.358850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:13.358850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:13.358850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:13.358850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:13.358850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:13.358850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:13.358850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:11:13.360317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:11:13.360317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:11:13.360317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:11:13.360317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 472/942] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:13.364829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd141abd-cb59-42fa-a2cb-12dd9bcede3b None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:13.367746 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:13.491531 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd141abd-cb59-42fa-a2cb-12dd9bcede3b None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:13.529109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85054b-a945-4df3-988f-1c38646d86ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:13.530057 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85054b-a945-4df3-988f-1c38646d86ed None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:13.530563 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85054b-a945-4df3-988f-1c38646d86ed None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:13.530865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85054b-a945-4df3-988f-1c38646d86ed None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:13.531760 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85054b-a945-4df3-988f-1c38646d86ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:13.532236 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85054b-a945-4df3-988f-1c38646d86ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:13.533833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85054b-a945-4df3-988f-1c38646d86ed None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:13.549412 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 471/943] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:11:13.555517 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82b6d9-3893-4908-8ff1-bfa6e5952881 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:13.603546 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:13.668719 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82b6d9-3893-4908-8ff1-bfa6e5952881 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:13.703907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cc60d-91e2-46f6-af01-b63ff1d1ba16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:13.704537 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cc60d-91e2-46f6-af01-b63ff1d1ba16 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:13.704827 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cc60d-91e2-46f6-af01-b63ff1d1ba16 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:13.705170 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cc60d-91e2-46f6-af01-b63ff1d1ba16 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:13.712690 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631cc60d-91e2-46f6-af01-b63ff1d1ba16 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:13.714115 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631cc60d-91e2-46f6-af01-b63ff1d1ba16 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:13.725507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-631cc60d-91e2-46f6-af01-b63ff1d1ba16 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 21 21:11:13.725507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:13.725507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:13.725507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:13.725507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:13.725507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:13.725507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:13.725507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:13.725507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:13.725507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:13.725507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:13.725507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:13.725507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:11:13.725507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:11:13.725507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:11:13.725507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:13.726612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:13.727547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:13.727547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:13.727547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:11:13.727547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:11:13.727547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:11:13.727547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:11:13.727547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:11:13.727547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:11:13.727547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 21 21:11:13.727547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:11:13.727547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 473/944] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:11:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:13.733258 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85054b-a945-4df3-988f-1c38646d86ed None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:13.798088 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:13.838907 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85054b-a945-4df3-988f-1c38646d86ed None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:13.873429 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3f286-cc5a-4fac-9db2-33f0cd70f17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:13.874028 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3f286-cc5a-4fac-9db2-33f0cd70f17c None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:13.874307 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3f286-cc5a-4fac-9db2-33f0cd70f17c None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:13.874578 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3f286-cc5a-4fac-9db2-33f0cd70f17c None admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:13.875343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3f286-cc5a-4fac-9db2-33f0cd70f17c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:13.875655 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3f286-cc5a-4fac-9db2-33f0cd70f17c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:13.877097 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3f286-cc5a-4fac-9db2-33f0cd70f17c None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:13.894391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 472/945] 10.222.0.13 () {56 vars in 1143 bytes} [Fri Oct 21 21:11:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:13.900683 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cc60d-91e2-46f6-af01-b63ff1d1ba16 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:14.005312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cc60d-91e2-46f6-af01-b63ff1d1ba16 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:14.006827 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:14.045955 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b148f-221b-499e-9844-43caf79a2351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:14.046682 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b148f-221b-499e-9844-43caf79a2351 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:14.046957 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b148f-221b-499e-9844-43caf79a2351 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:14.047154 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b148f-221b-499e-9844-43caf79a2351 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:14.058613 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b148f-221b-499e-9844-43caf79a2351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:14.060083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b148f-221b-499e-9844-43caf79a2351 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:14.074501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-693b148f-221b-499e-9844-43caf79a2351 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:11:14.074501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:14.074501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:14.074501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:14.074501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:14.074501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:14.074501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:14.074501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:14.074501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:14.074501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:14.074501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:14.074501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:14.074501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:11:14.074501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:11:14.074501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:11:14.074501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:11:14.075461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:14.075461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:14.075461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:14.075461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:14.075461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:14.075461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:14.075461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:14.075461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:14.075461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:14.075461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:14.075461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:14.075461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:14.075461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:14.075461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:14.075461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:14.075461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:14.075461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:11:14.076430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:11:14.076430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:11:14.076430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:11:14.076430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 474/946] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:14.081648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3f286-cc5a-4fac-9db2-33f0cd70f17c None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:14.187718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3f286-cc5a-4fac-9db2-33f0cd70f17c None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:14.206897 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:14.222042 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b91ab-1142-480e-b5f0-998cf433b667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:14.222687 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b91ab-1142-480e-b5f0-998cf433b667 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:14.222965 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b91ab-1142-480e-b5f0-998cf433b667 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:14.223289 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b91ab-1142-480e-b5f0-998cf433b667 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:14.223923 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b91ab-1142-480e-b5f0-998cf433b667 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:14.224223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b91ab-1142-480e-b5f0-998cf433b667 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:14.225600 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b91ab-1142-480e-b5f0-998cf433b667 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:14.238415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 473/947] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:14.243973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b148f-221b-499e-9844-43caf79a2351 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:14.357090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b148f-221b-499e-9844-43caf79a2351 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:14.393865 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a90dff-0e04-446b-9574-2aad20dd05ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:14.394503 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a90dff-0e04-446b-9574-2aad20dd05ac None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:14.394779 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a90dff-0e04-446b-9574-2aad20dd05ac None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:14.395054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a90dff-0e04-446b-9574-2aad20dd05ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:14.402513 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a90dff-0e04-446b-9574-2aad20dd05ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:14.403982 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a90dff-0e04-446b-9574-2aad20dd05ac None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:14.416423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-d5a90dff-0e04-446b-9574-2aad20dd05ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:11:14.416423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:14.416423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:14.416423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:14.416423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:14.416423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:14.416423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:14.416423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:14.416423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:14.416423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:14.416423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:14.416423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:14.416423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:11:14.416423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:11:14.416423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:11:14.417393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:11:14.417393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:14.417393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:14.417393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:14.417393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:14.417393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:14.417393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:14.417393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:14.417393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:14.417393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:14.417393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:14.417393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:14.417393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:14.417393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:14.417393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:14.417393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:14.417393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:14.418427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:11:14.418427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:11:14.418427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:11:14.418427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:11:14.418427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:11:14.418427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:11:14.418427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:11:14.418427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:11:14.418427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 475/948] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:11:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:14.423813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b91ab-1142-480e-b5f0-998cf433b667 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:14.557046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b91ab-1142-480e-b5f0-998cf433b667 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:14.596453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daade38-8370-45a3-aab1-7b9004555e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:14.597067 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daade38-8370-45a3-aab1-7b9004555e24 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:14.597348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daade38-8370-45a3-aab1-7b9004555e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:14.597624 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daade38-8370-45a3-aab1-7b9004555e24 None admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:14.598311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daade38-8370-45a3-aab1-7b9004555e24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:14.598688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daade38-8370-45a3-aab1-7b9004555e24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:14.600258 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daade38-8370-45a3-aab1-7b9004555e24 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:14.613351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 474/949] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:11:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:14.619455 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a90dff-0e04-446b-9574-2aad20dd05ac None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:14.636360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:14.724976 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a90dff-0e04-446b-9574-2aad20dd05ac None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:14.762255 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeef961-d7ac-433c-a5af-1ecd9dcda4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:14.762921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeef961-d7ac-433c-a5af-1ecd9dcda4cf None admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:14.763202 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeef961-d7ac-433c-a5af-1ecd9dcda4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:14.763496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeef961-d7ac-433c-a5af-1ecd9dcda4cf None admin] PATH_INFO: `/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/124f2b5b95e8450ea6867f6868b9e810/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:14.765489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeef961-d7ac-433c-a5af-1ecd9dcda4cf None admin] RBAC: Authorizing `identity:create_grant(project_id=8b2c33250ef84d79a4a4742fb4252156, user_id=124f2b5b95e8450ea6867f6868b9e810, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:14.766918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeef961-d7ac-433c-a5af-1ecd9dcda4cf None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:14.779658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 476/950] 10.222.0.13 () {58 vars in 1439 bytes} [Fri Oct 21 21:11:14 2022] PUT /identity/v3/projects/8b2c33250ef84d79a4a4742fb4252156/users/124f2b5b95e8450ea6867f6868b9e810/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:11:17.240918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ee71c-521c-4f91-af7c-3cccdce7fba3 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:17.241451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ee71c-521c-4f91-af7c-3cccdce7fba3 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:17.241451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ee71c-521c-4f91-af7c-3cccdce7fba3 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:17.244288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 475/951] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:11:17.248503 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35268215-f019-4bcf-93f7-154a52c057bd None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:17.248905 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35268215-f019-4bcf-93f7-154a52c057bd None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:17.249053 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35268215-f019-4bcf-93f7-154a52c057bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:17.268879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-35268215-f019-4bcf-93f7-154a52c057bd None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:11:17.270190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-35268215-f019-4bcf-93f7-154a52c057bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:11:17.342931 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 477/952] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:11:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:11:17.626405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ee71c-521c-4f91-af7c-3cccdce7fba3 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:17.681313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:17.775066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ee71c-521c-4f91-af7c-3cccdce7fba3 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:17.815175 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07dfed-7937-45bf-a740-bff432e0fd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:17.815888 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07dfed-7937-45bf-a740-bff432e0fd96 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:17.816137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07dfed-7937-45bf-a740-bff432e0fd96 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:17.816434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07dfed-7937-45bf-a740-bff432e0fd96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:17.827353 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07dfed-7937-45bf-a740-bff432e0fd96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 21:11:17.832279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-6f07dfed-7937-45bf-a740-bff432e0fd96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:11:17.832279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:17.832279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:17.832279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:17.832279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:17.832279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:17.832279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:17.832279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:17.832279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:17.832279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:17.832279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:17.832279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:17.832279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 21:11:17.832279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 21:11:17.832279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 21:11:17.832279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 21:11:17.833901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 21:11:17.833901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 21:11:17.833901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 21:11:17.833901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 21:11:17.833901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:17.833901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:17.833901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:17.833901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:17.833901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:17.833901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:17.833901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:17.833901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:17.833901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:17.833901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:17.833901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:17.833901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:17.835380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:17.835380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:17.835380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:17.835380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:17.835380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 21:11:17.835380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 21:11:17.835380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 21:11:17.835380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 21:11:17.835380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 21:11:17.835380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 21:11:17.835380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 21:11:17.835380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:17.835380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 476/953] 10.222.0.13 () {56 vars in 1096 bytes} [Fri Oct 21 21:11:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:17.839910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35268215-f019-4bcf-93f7-154a52c057bd None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:17.843342 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:17.951133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35268215-f019-4bcf-93f7-154a52c057bd None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:17.987335 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8b7cf-bda3-4660-a509-3d2c6939f4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:17.987992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8b7cf-bda3-4660-a509-3d2c6939f4ae None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:17.988292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8b7cf-bda3-4660-a509-3d2c6939f4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:17.988561 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8b7cf-bda3-4660-a509-3d2c6939f4ae None admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:17.989234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8b7cf-bda3-4660-a509-3d2c6939f4ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:17.989535 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8b7cf-bda3-4660-a509-3d2c6939f4ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:17.990918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8b7cf-bda3-4660-a509-3d2c6939f4ae None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:18.000128 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 478/954] 10.222.0.13 () {56 vars in 1099 bytes} [Fri Oct 21 21:11:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:18.006232 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07dfed-7937-45bf-a740-bff432e0fd96 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:18.106661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:18.109863 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07dfed-7937-45bf-a740-bff432e0fd96 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:18.145362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65762a9e-cabd-4de1-9d56-682f0c355d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:18.146008 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65762a9e-cabd-4de1-9d56-682f0c355d38 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:18.146287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65762a9e-cabd-4de1-9d56-682f0c355d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:18.146542 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65762a9e-cabd-4de1-9d56-682f0c355d38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:18.157701 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65762a9e-cabd-4de1-9d56-682f0c355d38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:18.159181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65762a9e-cabd-4de1-9d56-682f0c355d38 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:18.174520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-65762a9e-cabd-4de1-9d56-682f0c355d38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:11:18.174520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:18.174520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:18.174520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:18.174520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:18.174520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:18.174520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:18.174520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:18.174520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:18.174520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:18.174520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:18.174520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:18.174520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:11:18.174520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:11:18.174520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:11:18.174520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:11:18.175562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:18.175562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:18.175562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:18.175562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:18.175562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:18.175562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:18.175562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:18.175562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:18.175562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:18.175562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:18.175562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:18.175562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:18.175562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:18.175562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:18.175562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:18.175562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:18.175562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:11:18.176634 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:11:18.176634 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:11:18.176634 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:18.176634 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 477/955] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:18.181786 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8b7cf-bda3-4660-a509-3d2c6939f4ae None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:18.249218 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:18.286517 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8b7cf-bda3-4660-a509-3d2c6939f4ae None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:18.320859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75075534-abca-4c3c-8f33-62da725776e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:18.321995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75075534-abca-4c3c-8f33-62da725776e0 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:18.322273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75075534-abca-4c3c-8f33-62da725776e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:18.322545 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75075534-abca-4c3c-8f33-62da725776e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:18.323191 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75075534-abca-4c3c-8f33-62da725776e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:18.323493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75075534-abca-4c3c-8f33-62da725776e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:18.324873 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75075534-abca-4c3c-8f33-62da725776e0 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:18.336600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 479/956] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:18.342883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65762a9e-cabd-4de1-9d56-682f0c355d38 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:18.443803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65762a9e-cabd-4de1-9d56-682f0c355d38 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:18.481149 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc4141-c5a2-430d-af03-c95d087b26bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:18.481875 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc4141-c5a2-430d-af03-c95d087b26bf None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:18.482153 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc4141-c5a2-430d-af03-c95d087b26bf None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:18.482456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc4141-c5a2-430d-af03-c95d087b26bf None admin] PATH_INFO: `/v3/users/placement` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:18.490684 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc4141-c5a2-430d-af03-c95d087b26bf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:18.492166 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc4141-c5a2-430d-af03-c95d087b26bf None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:18.503981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-a4bc4141-c5a2-430d-af03-c95d087b26bf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 21 21:11:18.503981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:18.503981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:18.503981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:18.503981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:18.503981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:18.503981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:18.503981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:18.503981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:18.503981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:18.503981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:18.503981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:18.503981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 21:11:18.503981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 21:11:18.503981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 21:11:18.503981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:18.505118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:18.506491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:18.506491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:18.506491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 21:11:18.506491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 21:11:18.506491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 21:11:18.506491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 21:11:18.506491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 21:11:18.506491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 21:11:18.506491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 21 21:11:18.506491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:18.506491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 478/957] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:11:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:18.511701 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75075534-abca-4c3c-8f33-62da725776e0 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:18.609221 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75075534-abca-4c3c-8f33-62da725776e0 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:18.633895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:18.643477 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2b0a8-5935-4b62-b43f-150e4b73c0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:18.644103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2b0a8-5935-4b62-b43f-150e4b73c0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:18.644383 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2b0a8-5935-4b62-b43f-150e4b73c0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:18.644656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2b0a8-5935-4b62-b43f-150e4b73c0e3 None admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:18.645374 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd2b0a8-5935-4b62-b43f-150e4b73c0e3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:18.645704 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd2b0a8-5935-4b62-b43f-150e4b73c0e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:18.647106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd2b0a8-5935-4b62-b43f-150e4b73c0e3 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:18.663423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 480/958] 10.222.0.13 () {56 vars in 1143 bytes} [Fri Oct 21 21:11:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:18.669328 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc4141-c5a2-430d-af03-c95d087b26bf None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:18.684403 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:18.766111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc4141-c5a2-430d-af03-c95d087b26bf None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:18.797482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc352080-e8d2-444d-851d-ecc6ead403a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:18.798071 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc352080-e8d2-444d-851d-ecc6ead403a5 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:18.798343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc352080-e8d2-444d-851d-ecc6ead403a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:18.798608 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc352080-e8d2-444d-851d-ecc6ead403a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:18.860924 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc352080-e8d2-444d-851d-ecc6ead403a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:18.862344 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc352080-e8d2-444d-851d-ecc6ead403a5 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:18.875367 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:18.954363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 479/959] 10.222.0.13 () {58 vars in 1282 bytes} [Fri Oct 21 21:11:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 21:11:18.960123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2b0a8-5935-4b62-b43f-150e4b73c0e3 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:19.053687 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2b0a8-5935-4b62-b43f-150e4b73c0e3 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:19.085338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451400fb-e87f-47fc-87b3-5682286af4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:19.085944 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451400fb-e87f-47fc-87b3-5682286af4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:19.086218 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451400fb-e87f-47fc-87b3-5682286af4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:19.086485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451400fb-e87f-47fc-87b3-5682286af4f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:19.096364 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451400fb-e87f-47fc-87b3-5682286af4f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:19.097757 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451400fb-e87f-47fc-87b3-5682286af4f9 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:19.112291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-451400fb-e87f-47fc-87b3-5682286af4f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:11:19.112291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:19.112291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:19.112291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:19.112291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:19.112291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:19.112291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:19.112291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:19.112291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:19.112291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:19.112291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:19.112291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:19.112291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 21:11:19.112291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 21:11:19.112291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 21:11:19.112291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 21:11:19.113389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:19.113389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:19.113389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:19.113389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:19.113389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:19.113389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:19.113389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:19.113389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:19.113389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:19.113389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:19.113389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:19.113389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:19.113389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:19.113389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:19.113389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:19.113389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:19.113389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 21:11:19.114340 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 21:11:19.114340 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 21:11:19.114340 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:11:19.114340 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 481/960] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:19.119434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc352080-e8d2-444d-851d-ecc6ead403a5 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:19.212753 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc352080-e8d2-444d-851d-ecc6ead403a5 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:19.224948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:19.245590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75725d2-536b-4bee-9e29-bd8db5f73008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:19.246186 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75725d2-536b-4bee-9e29-bd8db5f73008 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:19.246452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75725d2-536b-4bee-9e29-bd8db5f73008 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:19.246720 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75725d2-536b-4bee-9e29-bd8db5f73008 None admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:19.247337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75725d2-536b-4bee-9e29-bd8db5f73008 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:19.247663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75725d2-536b-4bee-9e29-bd8db5f73008 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:19.249006 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75725d2-536b-4bee-9e29-bd8db5f73008 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:19.260346 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 480/961] 10.222.0.13 () {56 vars in 1109 bytes} [Fri Oct 21 21:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:19.265788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451400fb-e87f-47fc-87b3-5682286af4f9 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:19.299127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:19.360770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451400fb-e87f-47fc-87b3-5682286af4f9 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:19.393386 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051e4c9-9b79-4f0b-a068-05817ecf8617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:19.393990 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051e4c9-9b79-4f0b-a068-05817ecf8617 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:19.394269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051e4c9-9b79-4f0b-a068-05817ecf8617 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:19.394543 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051e4c9-9b79-4f0b-a068-05817ecf8617 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:19.400874 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051e4c9-9b79-4f0b-a068-05817ecf8617 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:19.402286 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051e4c9-9b79-4f0b-a068-05817ecf8617 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:19.413156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application [None req-c051e4c9-9b79-4f0b-a068-05817ecf8617 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:11:19.413156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:19.413156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:19.413156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:19.413156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:19.413156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:19.413156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:19.413156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:19.413156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:19.413156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:19.413156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:19.413156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:19.413156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 21:11:19.413156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 21:11:19.413156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 21:11:19.414153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 21:11:19.414153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:19.414153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:19.414153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:19.414153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:19.414153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:19.414153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:19.414153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:19.414153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:19.414153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:19.414153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:19.414153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:19.414153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:19.414153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:19.414153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:19.414153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:19.414153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:19.415244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 21:11:19.415244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 21:11:19.415244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 21:11:19.415244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 21:11:19.415244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 21:11:19.415244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 21:11:19.415244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 21:11:19.415244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: ERROR keystone.server.flask.application Oct 21 21:11:19.415244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 482/962] 10.222.0.13 () {56 vars in 1111 bytes} [Fri Oct 21 21:11:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:19.420094 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75725d2-536b-4bee-9e29-bd8db5f73008 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:19.525227 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75725d2-536b-4bee-9e29-bd8db5f73008 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:19.557314 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e2e2b-4311-4988-9fbf-aea44cf4956f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:19.557914 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e2e2b-4311-4988-9fbf-aea44cf4956f None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:19.558191 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e2e2b-4311-4988-9fbf-aea44cf4956f None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:19.558459 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e2e2b-4311-4988-9fbf-aea44cf4956f None admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:19.559131 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e2e2b-4311-4988-9fbf-aea44cf4956f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:19.559432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e2e2b-4311-4988-9fbf-aea44cf4956f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:19.560882 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e2e2b-4311-4988-9fbf-aea44cf4956f None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:19.572170 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 481/963] 10.222.0.13 () {56 vars in 1148 bytes} [Fri Oct 21 21:11:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:19.578386 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051e4c9-9b79-4f0b-a068-05817ecf8617 None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:19.668317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:19.674332 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051e4c9-9b79-4f0b-a068-05817ecf8617 None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:19.706019 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dbe0c-f493-435a-92c7-3d0b3ce47cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:19.706595 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dbe0c-f493-435a-92c7-3d0b3ce47cbf None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:19.706874 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dbe0c-f493-435a-92c7-3d0b3ce47cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:19.707141 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dbe0c-f493-435a-92c7-3d0b3ce47cbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:19.707876 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4dbe0c-f493-435a-92c7-3d0b3ce47cbf None admin] RBAC: Adding query filter params (role.id=c514b065b7604094b899c586f79b0c5b, scope.project.id=8b2c33250ef84d79a4a4742fb4252156, user.id=124f2b5b95e8450ea6867f6868b9e810) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:19.708180 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4dbe0c-f493-435a-92c7-3d0b3ce47cbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:19.709572 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4dbe0c-f493-435a-92c7-3d0b3ce47cbf None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:19.715044 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 483/964] 10.222.0.13 () {56 vars in 1374 bytes} [Fri Oct 21 21:11:19 2022] GET /identity/v3/role_assignments?user.id=124f2b5b95e8450ea6867f6868b9e810&scope.project.id=8b2c33250ef84d79a4a4742fb4252156&role.id=c514b065b7604094b899c586f79b0c5b => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:21.841447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b19b8-1a09-48e9-aff8-3d38d8bda5fa None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:21.841953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b19b8-1a09-48e9-aff8-3d38d8bda5fa None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:21.842044 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b19b8-1a09-48e9-aff8-3d38d8bda5fa None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:21.843315 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 482/965] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:11:21.848585 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d99be-327a-4d86-813b-989acc57dd7c None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:21.848948 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d99be-327a-4d86-813b-989acc57dd7c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:21.849142 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d99be-327a-4d86-813b-989acc57dd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:21.866782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-271d99be-327a-4d86-813b-989acc57dd7c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:11:21.868004 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-271d99be-327a-4d86-813b-989acc57dd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:11:21.931813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 484/966] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:11:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:11:22.201117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b19b8-1a09-48e9-aff8-3d38d8bda5fa None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:22.231337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:22.297859 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b19b8-1a09-48e9-aff8-3d38d8bda5fa None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:22.331895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705a440-99ec-46c1-a7b4-76a86b518f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:22.332510 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705a440-99ec-46c1-a7b4-76a86b518f87 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:22.332782 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705a440-99ec-46c1-a7b4-76a86b518f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:22.333048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705a440-99ec-46c1-a7b4-76a86b518f87 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:22.333609 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705a440-99ec-46c1-a7b4-76a86b518f87 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:22.334965 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705a440-99ec-46c1-a7b4-76a86b518f87 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:22.343284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-b705a440-99ec-46c1-a7b4-76a86b518f87 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 21:11:22.343284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:22.343284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:11:22.343284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:11:22.343284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:11:22.343284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:11:22.343284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:11:22.343284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:11:22.343284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 21:11:22.343284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:22.343284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:11:22.343284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:22.343284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:22.343284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:22.343284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:22.343284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:22.343284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:22.344555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:22.344555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:22.344555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:22.344555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:22.344555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:22.344555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:22.344555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:11:22.344555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:11:22.344555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:11:22.344555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:11:22.344555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:22.344555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:22.344555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:22.344555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:22.344555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:22.344555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:22.345773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:22.345773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:22.345773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:22.345773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:22.345773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:22.345773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:22.345773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:22.345773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:22.345773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:22.345773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:22.345773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:11:22.345773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:11:22.345773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 21:11:22.345773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:22.345773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 483/967] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Oct 21 21:11:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:22.350341 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d99be-327a-4d86-813b-989acc57dd7c None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:22.436006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:22.447825 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d99be-327a-4d86-813b-989acc57dd7c None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:22.488020 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359d0e7-3240-4c54-871e-0f0635973eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:22.488739 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359d0e7-3240-4c54-871e-0f0635973eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:22.489096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359d0e7-3240-4c54-871e-0f0635973eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:22.489401 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359d0e7-3240-4c54-871e-0f0635973eb9 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:22.490140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359d0e7-3240-4c54-871e-0f0635973eb9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:22.490520 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359d0e7-3240-4c54-871e-0f0635973eb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:22.492272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4359d0e7-3240-4c54-871e-0f0635973eb9 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:22.498142 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 485/968] 10.222.0.13 () {56 vars in 1116 bytes} [Fri Oct 21 21:11:22 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:22.504896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705a440-99ec-46c1-a7b4-76a86b518f87 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:22.594258 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:22.600088 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705a440-99ec-46c1-a7b4-76a86b518f87 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:22.632198 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d4c65-a27b-4e01-be4c-892545993357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:22.632792 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d4c65-a27b-4e01-be4c-892545993357 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:22.633067 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d4c65-a27b-4e01-be4c-892545993357 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:22.633337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d4c65-a27b-4e01-be4c-892545993357 None admin] PATH_INFO: `/v3/services` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:22.634228 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d4c65-a27b-4e01-be4c-892545993357 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:22.634530 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d4c65-a27b-4e01-be4c-892545993357 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:22.635927 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d4c65-a27b-4e01-be4c-892545993357 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:22.640116 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 484/969] 10.222.0.13 () {56 vars in 1116 bytes} [Fri Oct 21 21:11:22 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:24.733919 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73c94e-88fb-45ff-b6c8-7d9c58b10f20 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:24.734436 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73c94e-88fb-45ff-b6c8-7d9c58b10f20 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:24.734436 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73c94e-88fb-45ff-b6c8-7d9c58b10f20 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:24.735824 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 486/970] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:11:24.741237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8b6f7-eb3e-468b-9cc3-d1ff9d6946fd None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:24.741629 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8b6f7-eb3e-468b-9cc3-d1ff9d6946fd None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:24.741784 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8b6f7-eb3e-468b-9cc3-d1ff9d6946fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:24.760260 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-0fa8b6f7-eb3e-468b-9cc3-d1ff9d6946fd None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:11:24.761433 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0fa8b6f7-eb3e-468b-9cc3-d1ff9d6946fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:11:24.824415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 485/971] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:11:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:11:25.090098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73c94e-88fb-45ff-b6c8-7d9c58b10f20 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:25.186458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73c94e-88fb-45ff-b6c8-7d9c58b10f20 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:25.218396 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bd597-55b4-4394-8f91-f737b27e6c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:25.218983 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bd597-55b4-4394-8f91-f737b27e6c11 None admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:25.219261 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bd597-55b4-4394-8f91-f737b27e6c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:25.219531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bd597-55b4-4394-8f91-f737b27e6c11 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:25.220209 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bd597-55b4-4394-8f91-f737b27e6c11 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:25.221543 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bd597-55b4-4394-8f91-f737b27e6c11 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:25.236191 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 487/972] 10.222.0.13 () {60 vars in 1141 bytes} [Fri Oct 21 21:11:25 2022] POST /identity/v3/services => generated 246 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:11:27.376889 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45567b1f-9f87-4fc1-9724-3551947e2770 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:27.377417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45567b1f-9f87-4fc1-9724-3551947e2770 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:27.377417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45567b1f-9f87-4fc1-9724-3551947e2770 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:27.378747 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 486/973] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:11:27.384309 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdec4f-7009-485b-a286-076f4ab2af2e None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:27.384710 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdec4f-7009-485b-a286-076f4ab2af2e None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:27.384858 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdec4f-7009-485b-a286-076f4ab2af2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:27.403753 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-95bdec4f-7009-485b-a286-076f4ab2af2e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:11:27.404965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-95bdec4f-7009-485b-a286-076f4ab2af2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:11:27.413901 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:27.482278 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 488/974] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:11:27 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:11:27.747867 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45567b1f-9f87-4fc1-9724-3551947e2770 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:27.833665 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:27.844615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45567b1f-9f87-4fc1-9724-3551947e2770 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:27.876354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed06b21-4a37-4255-9806-2bca2d249277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:27.876980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed06b21-4a37-4255-9806-2bca2d249277 None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:27.877257 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed06b21-4a37-4255-9806-2bca2d249277 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:27.877528 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed06b21-4a37-4255-9806-2bca2d249277 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:27.878097 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed06b21-4a37-4255-9806-2bca2d249277 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:27.879485 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed06b21-4a37-4255-9806-2bca2d249277 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:27.888688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-4ed06b21-4a37-4255-9806-2bca2d249277 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 21:11:27.888688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:27.888688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:11:27.888688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:11:27.888688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:11:27.888688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:11:27.888688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:11:27.888688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:11:27.888688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 21:11:27.888688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:27.888688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:11:27.888688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:27.888688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:27.888688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:27.888688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:27.888688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:27.888688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:27.889812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:27.889812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:27.889812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:27.889812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:27.889812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:27.889812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:27.889812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:11:27.889812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:11:27.889812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:11:27.889812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:11:27.889812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:27.889812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:27.889812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:27.889812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:27.889812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:27.889812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:27.890783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:27.890783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:27.890783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:27.890783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:27.890783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:27.890783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:27.890783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:27.890783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:27.890783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:27.890783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:27.890783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:11:27.890783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:11:27.890783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 21:11:27.890783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:27.890783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 487/975] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Oct 21 21:11:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:27.895648 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdec4f-7009-485b-a286-076f4ab2af2e None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:27.990234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdec4f-7009-485b-a286-076f4ab2af2e None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:28.022548 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8074d-d653-4f50-9797-491d533b717d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:28.023140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8074d-d653-4f50-9797-491d533b717d None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:28.023421 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8074d-d653-4f50-9797-491d533b717d None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:28.023725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8074d-d653-4f50-9797-491d533b717d None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:28.024406 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8074d-d653-4f50-9797-491d533b717d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:28.024707 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8074d-d653-4f50-9797-491d533b717d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:28.026056 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8074d-d653-4f50-9797-491d533b717d None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:28.031085 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 489/976] 10.222.0.13 () {56 vars in 1116 bytes} [Fri Oct 21 21:11:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:28.036741 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed06b21-4a37-4255-9806-2bca2d249277 None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:28.131606 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed06b21-4a37-4255-9806-2bca2d249277 None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:28.163558 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63785462-d136-456a-9dca-dfdd1839904e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:28.164179 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63785462-d136-456a-9dca-dfdd1839904e None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:28.164458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63785462-d136-456a-9dca-dfdd1839904e None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:28.165293 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63785462-d136-456a-9dca-dfdd1839904e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:28.165984 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63785462-d136-456a-9dca-dfdd1839904e None admin] RBAC: Adding query filter params (interface=public, service_id=698551f7a2a04036b6a26274beb3245d, region_id=RegionOne) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:28.166291 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63785462-d136-456a-9dca-dfdd1839904e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:28.167668 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63785462-d136-456a-9dca-dfdd1839904e None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:28.172504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 488/977] 10.222.0.13 () {56 vars in 1251 bytes} [Fri Oct 21 21:11:28 2022] GET /identity/v3/endpoints?service_id=698551f7a2a04036b6a26274beb3245d&interface=public®ion_id=RegionOne => generated 194 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:28.177901 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8074d-d653-4f50-9797-491d533b717d None admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:28.181560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:28.271784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8074d-d653-4f50-9797-491d533b717d None admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:28.302783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffea0a4-9e2e-4cbc-a517-dbf038ba7708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:28.303363 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffea0a4-9e2e-4cbc-a517-dbf038ba7708 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:28.303665 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffea0a4-9e2e-4cbc-a517-dbf038ba7708 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:28.303934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffea0a4-9e2e-4cbc-a517-dbf038ba7708 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:28.304519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffea0a4-9e2e-4cbc-a517-dbf038ba7708 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:28.306346 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffea0a4-9e2e-4cbc-a517-dbf038ba7708 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:28.312857 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 490/978] 10.222.0.13 () {56 vars in 1087 bytes} [Fri Oct 21 21:11:28 2022] GET /identity/v3/services => generated 2191 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:11:30.437660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca593cd-19ff-43f5-94c8-b27afea26854 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:30.438188 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca593cd-19ff-43f5-94c8-b27afea26854 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:30.438188 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca593cd-19ff-43f5-94c8-b27afea26854 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:30.439534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 489/979] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:11:30.445130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d95a5-a302-4eeb-95d7-0c80a185a004 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:30.445540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d95a5-a302-4eeb-95d7-0c80a185a004 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:30.445739 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d95a5-a302-4eeb-95d7-0c80a185a004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:30.466741 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-f10d95a5-a302-4eeb-95d7-0c80a185a004 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:11:30.468171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-f10d95a5-a302-4eeb-95d7-0c80a185a004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:11:30.534039 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:30.540078 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 491/980] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:11:30 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 21:11:30.811286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca593cd-19ff-43f5-94c8-b27afea26854 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:30.830500 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:30.908403 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca593cd-19ff-43f5-94c8-b27afea26854 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:30.939264 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86ad9f-9cc9-4911-9b03-27460255f38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:30.939905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86ad9f-9cc9-4911-9b03-27460255f38a None admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:30.940183 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86ad9f-9cc9-4911-9b03-27460255f38a None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:30.940453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86ad9f-9cc9-4911-9b03-27460255f38a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:30.941017 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86ad9f-9cc9-4911-9b03-27460255f38a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:30.942384 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86ad9f-9cc9-4911-9b03-27460255f38a None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:30.951294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application [None req-2b86ad9f-9cc9-4911-9b03-27460255f38a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 21:11:30.951294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:30.951294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 21:11:30.951294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 21:11:30.951294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 21:11:30.951294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 21:11:30.951294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 21:11:30.951294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:11:30.951294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 21:11:30.951294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:30.951294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 21:11:30.951294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:30.951294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 21:11:30.951294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 21:11:30.951294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 21:11:30.951294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 21:11:30.951294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 21:11:30.952416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 21:11:30.952416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 21:11:30.952416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 21:11:30.952416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 21:11:30.952416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 21:11:30.952416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 21:11:30.952416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 21:11:30.952416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 21:11:30.952416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 21:11:30.952416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 21:11:30.952416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 21:11:30.952416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 21:11:30.952416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 21:11:30.952416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 21:11:30.952416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 21:11:30.952416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 21:11:30.953337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 21:11:30.953337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application with Lock( Oct 21 21:11:30.953337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 21:11:30.953337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self._enter() Oct 21 21:11:30.953337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 21:11:30.953337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 21:11:30.953337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 21:11:30.953337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application return self.creator() Oct 21 21:11:30.953337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 21:11:30.953337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application created_value = creator( Oct 21 21:11:30.953337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 21:11:30.953337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 21:11:30.953337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 21:11:30.953337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: ERROR keystone.server.flask.application Oct 21 21:11:30.953337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 490/981] 10.222.0.13 () {56 vars in 1117 bytes} [Fri Oct 21 21:11:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 21:11:30.958257 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d95a5-a302-4eeb-95d7-0c80a185a004 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:31.051670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d95a5-a302-4eeb-95d7-0c80a185a004 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:31.083836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2d251-ebe9-4051-974e-0aad0b204244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:31.084442 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2d251-ebe9-4051-974e-0aad0b204244 None admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:31.084715 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2d251-ebe9-4051-974e-0aad0b204244 None admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:31.084982 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2d251-ebe9-4051-974e-0aad0b204244 None admin] PATH_INFO: `/v3/services` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:31.085587 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2d251-ebe9-4051-974e-0aad0b204244 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:11:31.085883 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2d251-ebe9-4051-974e-0aad0b204244 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:31.087784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2d251-ebe9-4051-974e-0aad0b204244 None admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:31.092644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 492/982] 10.222.0.13 () {56 vars in 1116 bytes} [Fri Oct 21 21:11:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:11:31.098186 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86ad9f-9cc9-4911-9b03-27460255f38a None admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:11:31.176637 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:31.193744 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86ad9f-9cc9-4911-9b03-27460255f38a None admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:11:31.225325 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35566d2d-8a6e-46d5-a1a7-254c116023d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:11:31.225909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35566d2d-8a6e-46d5-a1a7-254c116023d6 None admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:11:31.226179 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35566d2d-8a6e-46d5-a1a7-254c116023d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:11:31.226446 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35566d2d-8a6e-46d5-a1a7-254c116023d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:11:31.227652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35566d2d-8a6e-46d5-a1a7-254c116023d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:11:31.229029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35566d2d-8a6e-46d5-a1a7-254c116023d6 None admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:11:31.247050 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 491/983] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:11:31 2022] POST /identity/v3/endpoints => generated 331 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:11:41.090305 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=69 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:11:41.242894 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:12:34.982736 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7338ee1-5300-4f47-a6cf-d6c6978e8a9a None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:34.983869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7338ee1-5300-4f47-a6cf-d6c6978e8a9a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:34.984649 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7338ee1-5300-4f47-a6cf-d6c6978e8a9a None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:34.993351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57af24-cd2a-4c92-b818-217c61d97f4c None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:34.993808 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57af24-cd2a-4c92-b818-217c61d97f4c None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:34.993889 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57af24-cd2a-4c92-b818-217c61d97f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:34.997366 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 493/984] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:12:34 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:12:35.095975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-8b57af24-cd2a-4c92-b818-217c61d97f4c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:12:35.097567 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8b57af24-cd2a-4c92-b818-217c61d97f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:35.201719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 492/985] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:12:35.496565 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264daa8b-a59a-4909-8c88-17d3e2234e7a None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:35.497133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264daa8b-a59a-4909-8c88-17d3e2234e7a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:35.497524 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264daa8b-a59a-4909-8c88-17d3e2234e7a None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:35.499106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 494/986] 10.222.0.13 () {54 vars in 964 bytes} [Fri Oct 21 21:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:12:35.505407 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de6ab8-97cd-4050-a258-b750a7efbdce None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:35.505764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de6ab8-97cd-4050-a258-b750a7efbdce None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:35.506106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de6ab8-97cd-4050-a258-b750a7efbdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:35.597127 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-88de6ab8-97cd-4050-a258-b750a7efbdce None None] MFA Rules not processed for user `d34dc4e7c5474b6b96977019328d8890`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:12:35.598415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-88de6ab8-97cd-4050-a258-b750a7efbdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:35.622449 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:12:35.688364 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 493/987] 10.222.0.13 () {58 vars in 1061 bytes} [Fri Oct 21 21:12:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:12:35.697715 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264daa8b-a59a-4909-8c88-17d3e2234e7a None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:12:35.699483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-264daa8b-a59a-4909-8c88-17d3e2234e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:35.820442 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264daa8b-a59a-4909-8c88-17d3e2234e7a None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:12:35.900324 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c16d64-9679-412b-936c-6279ad069d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:12:35.900933 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c16d64-9679-412b-936c-6279ad069d22 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:35.901208 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c16d64-9679-412b-936c-6279ad069d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:35.901474 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c16d64-9679-412b-936c-6279ad069d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:35.903053 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-10c16d64-9679-412b-936c-6279ad069d22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:35.961227 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:12:36.019089 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c16d64-9679-412b-936c-6279ad069d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:12:36.020533 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c16d64-9679-412b-936c-6279ad069d22 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:12:36.041905 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 495/988] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:12:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:12:38.474142 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93ec96-0f05-4c9f-b360-384518cf1661 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:38.474656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93ec96-0f05-4c9f-b360-384518cf1661 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:38.474656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93ec96-0f05-4c9f-b360-384518cf1661 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:38.476070 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 494/989] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:12:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:12:38.481291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ca4ec-c64b-4e2b-af16-6ddf55ae059e None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:38.481680 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ca4ec-c64b-4e2b-af16-6ddf55ae059e None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:38.481833 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ca4ec-c64b-4e2b-af16-6ddf55ae059e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:38.565702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-fc4ca4ec-c64b-4e2b-af16-6ddf55ae059e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:12:38.566855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-fc4ca4ec-c64b-4e2b-af16-6ddf55ae059e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:38.660907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 496/990] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:12:38.942529 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93ec96-0f05-4c9f-b360-384518cf1661 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:12:38.944099 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-3b93ec96-0f05-4c9f-b360-384518cf1661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:39.044656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93ec96-0f05-4c9f-b360-384518cf1661 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:12:39.115491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:12:39.117813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e047e9-43ab-4829-a4d4-ca674af663ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:12:39.118624 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e047e9-43ab-4829-a4d4-ca674af663ad service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:39.118921 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e047e9-43ab-4829-a4d4-ca674af663ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:39.119337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e047e9-43ab-4829-a4d4-ca674af663ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:39.121200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b1e047e9-43ab-4829-a4d4-ca674af663ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:39.232865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e047e9-43ab-4829-a4d4-ca674af663ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:12:39.234307 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e047e9-43ab-4829-a4d4-ca674af663ad service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:12:39.254224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 495/991] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:12:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:12:41.730061 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf720c6-ea60-4e9a-90de-2115026b3a1f None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:41.730584 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf720c6-ea60-4e9a-90de-2115026b3a1f None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:41.730584 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf720c6-ea60-4e9a-90de-2115026b3a1f None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:41.732027 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 497/992] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:12:41.737376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17103b51-98f6-4494-9f2e-d45e84bc25ba None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:41.737753 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17103b51-98f6-4494-9f2e-d45e84bc25ba None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:41.737975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17103b51-98f6-4494-9f2e-d45e84bc25ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:41.804303 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-17103b51-98f6-4494-9f2e-d45e84bc25ba None None] MFA Rules not processed for user `8e8963c7ee674a639acccd9a093fe9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:12:41.805513 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-17103b51-98f6-4494-9f2e-d45e84bc25ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:41.856427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:12:41.901344 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 496/993] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:12:41 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:12:42.185595 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf720c6-ea60-4e9a-90de-2115026b3a1f None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:12:42.187133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-eaf720c6-ea60-4e9a-90de-2115026b3a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:42.199834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:12:42.291584 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf720c6-ea60-4e9a-90de-2115026b3a1f None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:12:42.363732 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29249b-604a-4c30-a000-24921b8028be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:12:42.364340 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29249b-604a-4c30-a000-24921b8028be service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:42.364614 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29249b-604a-4c30-a000-24921b8028be service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:42.364883 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29249b-604a-4c30-a000-24921b8028be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:42.366491 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-cd29249b-604a-4c30-a000-24921b8028be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:42.426235 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:12:42.471986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29249b-604a-4c30-a000-24921b8028be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:12:42.473438 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29249b-604a-4c30-a000-24921b8028be service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:12:42.492262 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 498/994] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:12:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:12:45.176115 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735de884-3e80-4cdb-ac48-cc10a031075c None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:45.176617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735de884-3e80-4cdb-ac48-cc10a031075c None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:45.176732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735de884-3e80-4cdb-ac48-cc10a031075c None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:45.178034 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 497/995] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:12:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:12:45.183433 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfacc4-1a7d-4d3e-b227-be1f71f77826 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:45.183834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfacc4-1a7d-4d3e-b227-be1f71f77826 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:45.184006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfacc4-1a7d-4d3e-b227-be1f71f77826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:45.249982 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-b8bfacc4-1a7d-4d3e-b227-be1f71f77826 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:12:45.251195 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-b8bfacc4-1a7d-4d3e-b227-be1f71f77826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:45.345867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 499/996] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:12:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:12:45.636446 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735de884-3e80-4cdb-ac48-cc10a031075c None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:12:45.637980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-735de884-3e80-4cdb-ac48-cc10a031075c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:45.745468 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735de884-3e80-4cdb-ac48-cc10a031075c None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:12:45.823123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cbca32-5e18-4fe9-a3d1-ddf4ccf6fef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:12:45.823891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbca32-5e18-4fe9-a3d1-ddf4ccf6fef4 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:45.824250 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbca32-5e18-4fe9-a3d1-ddf4ccf6fef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:45.824614 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbca32-5e18-4fe9-a3d1-ddf4ccf6fef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:45.826462 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e1cbca32-5e18-4fe9-a3d1-ddf4ccf6fef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:45.834038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:12:45.932391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cbca32-5e18-4fe9-a3d1-ddf4ccf6fef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:12:45.933753 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cbca32-5e18-4fe9-a3d1-ddf4ccf6fef4 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:12:45.952776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 498/997] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:12:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:12:48.749312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da14f3-5795-47d0-b585-cb6b0c70679d None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:48.749806 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da14f3-5795-47d0-b585-cb6b0c70679d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:48.749893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da14f3-5795-47d0-b585-cb6b0c70679d None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:48.751211 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 500/998] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:12:48.757346 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c6d37-c3a0-421b-969b-adada8c6ee52 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:48.757792 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c6d37-c3a0-421b-969b-adada8c6ee52 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:48.757935 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c6d37-c3a0-421b-969b-adada8c6ee52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:48.826935 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-c48c6d37-c3a0-421b-969b-adada8c6ee52 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:12:48.828228 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-c48c6d37-c3a0-421b-969b-adada8c6ee52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:48.891008 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:12:48.933100 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 499/999] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:12:49.238226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da14f3-5795-47d0-b585-cb6b0c70679d None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:12:49.239916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-43da14f3-5795-47d0-b585-cb6b0c70679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:49.260798 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:12:49.349272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da14f3-5795-47d0-b585-cb6b0c70679d None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:12:49.426503 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84ecb1-0e60-4d40-8b44-74999b47e90a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:12:49.427112 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84ecb1-0e60-4d40-8b44-74999b47e90a service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:49.427389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84ecb1-0e60-4d40-8b44-74999b47e90a service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:49.427688 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84ecb1-0e60-4d40-8b44-74999b47e90a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:49.429283 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-7e84ecb1-0e60-4d40-8b44-74999b47e90a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:49.508416 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:12:49.548858 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84ecb1-0e60-4d40-8b44-74999b47e90a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:12:49.550313 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84ecb1-0e60-4d40-8b44-74999b47e90a service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:12:49.568609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 501/1000] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:12:51.793454 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab7597-dcc2-48d4-9a53-ce357393a480 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:51.793976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab7597-dcc2-48d4-9a53-ce357393a480 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:51.793976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab7597-dcc2-48d4-9a53-ce357393a480 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:51.795335 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 500/1001] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:12:51.800553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2514f-ee0d-4481-be84-993d029edcad None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:51.800999 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2514f-ee0d-4481-be84-993d029edcad None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:51.801207 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2514f-ee0d-4481-be84-993d029edcad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:51.868426 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-35c2514f-ee0d-4481-be84-993d029edcad None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:12:51.869621 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-35c2514f-ee0d-4481-be84-993d029edcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:51.973038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 502/1002] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:12:52.254880 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab7597-dcc2-48d4-9a53-ce357393a480 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:12:52.256764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-18ab7597-dcc2-48d4-9a53-ce357393a480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:52.358943 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab7597-dcc2-48d4-9a53-ce357393a480 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:12:52.436428 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c1878-65e2-42e9-b689-b0ff4804aeb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:12:52.437144 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c1878-65e2-42e9-b689-b0ff4804aeb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:52.437426 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c1878-65e2-42e9-b689-b0ff4804aeb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:52.437698 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c1878-65e2-42e9-b689-b0ff4804aeb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:52.439281 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d98c1878-65e2-42e9-b689-b0ff4804aeb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:52.457846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:12:52.558370 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c1878-65e2-42e9-b689-b0ff4804aeb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:12:52.559783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c1878-65e2-42e9-b689-b0ff4804aeb2 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:12:52.579403 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 501/1003] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:12:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:12:54.893104 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61b276-5436-46e4-a886-6c83da3ff601 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:54.893597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61b276-5436-46e4-a886-6c83da3ff601 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:54.893597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61b276-5436-46e4-a886-6c83da3ff601 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:54.895523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 503/1004] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:12:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:12:54.901012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b22c8-9de0-4a86-8246-5224caf847b2 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:54.901401 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b22c8-9de0-4a86-8246-5224caf847b2 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:54.901557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b22c8-9de0-4a86-8246-5224caf847b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:54.966608 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-dd0b22c8-9de0-4a86-8246-5224caf847b2 None None] MFA Rules not processed for user `8e8963c7ee674a639acccd9a093fe9ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:12:54.967843 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-dd0b22c8-9de0-4a86-8246-5224caf847b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:55.031395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:12:55.063330 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 502/1005] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:12:54 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:12:55.353892 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61b276-5436-46e4-a886-6c83da3ff601 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:12:55.355472 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-db61b276-5436-46e4-a886-6c83da3ff601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:55.387192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:12:55.460991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61b276-5436-46e4-a886-6c83da3ff601 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:12:55.536232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c238688-6499-4974-8319-6485a80f4de9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:12:55.536837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c238688-6499-4974-8319-6485a80f4de9 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:55.537115 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c238688-6499-4974-8319-6485a80f4de9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:55.537447 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c238688-6499-4974-8319-6485a80f4de9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:55.539044 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9c238688-6499-4974-8319-6485a80f4de9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:55.615374 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:12:55.648229 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c238688-6499-4974-8319-6485a80f4de9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:12:55.649981 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c238688-6499-4974-8319-6485a80f4de9 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:12:55.667972 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 504/1006] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:12:55 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:12:58.101889 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29cf4f-7191-4a7e-9ad9-62dffef67257 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:58.102409 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29cf4f-7191-4a7e-9ad9-62dffef67257 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:58.102409 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29cf4f-7191-4a7e-9ad9-62dffef67257 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:58.103712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 503/1007] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:12:58.108789 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda27dcb-f778-4e35-b61a-912e34b9b56d None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:58.109229 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda27dcb-f778-4e35-b61a-912e34b9b56d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:58.109333 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda27dcb-f778-4e35-b61a-912e34b9b56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:58.174442 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-eda27dcb-f778-4e35-b61a-912e34b9b56d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:12:58.176177 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-eda27dcb-f778-4e35-b61a-912e34b9b56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:58.272007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 505/1008] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:12:58.552644 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29cf4f-7191-4a7e-9ad9-62dffef67257 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:12:58.554192 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-4e29cf4f-7191-4a7e-9ad9-62dffef67257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:58.655458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29cf4f-7191-4a7e-9ad9-62dffef67257 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:12:58.728911 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08290edd-bf59-4bf1-8d3c-34fbb22d64b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:12:58.729517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08290edd-bf59-4bf1-8d3c-34fbb22d64b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:12:58.729793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08290edd-bf59-4bf1-8d3c-34fbb22d64b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:12:58.730062 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08290edd-bf59-4bf1-8d3c-34fbb22d64b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:12:58.731733 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-08290edd-bf59-4bf1-8d3c-34fbb22d64b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:12:58.758836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:12:58.838622 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08290edd-bf59-4bf1-8d3c-34fbb22d64b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:12:58.840025 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08290edd-bf59-4bf1-8d3c-34fbb22d64b9 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:12:58.858440 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 504/1009] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:12:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:13:01.032430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233477c2-fdae-4744-843d-b7d672253ebd None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:01.032931 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233477c2-fdae-4744-843d-b7d672253ebd None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:01.032931 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233477c2-fdae-4744-843d-b7d672253ebd None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:01.034268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 506/1010] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:01.039474 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d96eb-08cb-4b94-8a3f-266f059b0d77 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:01.039911 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d96eb-08cb-4b94-8a3f-266f059b0d77 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:01.040056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d96eb-08cb-4b94-8a3f-266f059b0d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:01.104211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-2e0d96eb-08cb-4b94-8a3f-266f059b0d77 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:01.105416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2e0d96eb-08cb-4b94-8a3f-266f059b0d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:01.178481 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:01.202246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 505/1011] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:01.491939 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233477c2-fdae-4744-843d-b7d672253ebd None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:13:01.493804 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-233477c2-fdae-4744-843d-b7d672253ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:01.535387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:01.598287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233477c2-fdae-4744-843d-b7d672253ebd None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:13:01.671581 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bacaa7-fdee-4b15-8540-bb72f14ea488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:13:01.672199 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bacaa7-fdee-4b15-8540-bb72f14ea488 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:01.672478 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bacaa7-fdee-4b15-8540-bb72f14ea488 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:01.672741 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bacaa7-fdee-4b15-8540-bb72f14ea488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:01.674306 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-18bacaa7-fdee-4b15-8540-bb72f14ea488 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:01.759096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:01.784790 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bacaa7-fdee-4b15-8540-bb72f14ea488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:13:01.786214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bacaa7-fdee-4b15-8540-bb72f14ea488 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:13:01.804273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 507/1012] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:13:04.293443 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac062f-6230-4f33-9f02-986f1a99593a None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:04.293994 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac062f-6230-4f33-9f02-986f1a99593a None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:04.293994 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac062f-6230-4f33-9f02-986f1a99593a None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:04.295360 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 506/1013] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:04.301099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a12c7-a3b7-4fd1-9577-8b316728139d None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:04.301545 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a12c7-a3b7-4fd1-9577-8b316728139d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:04.301646 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a12c7-a3b7-4fd1-9577-8b316728139d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:04.366285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-659a12c7-a3b7-4fd1-9577-8b316728139d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:04.367407 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-659a12c7-a3b7-4fd1-9577-8b316728139d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:04.464178 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 508/1014] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:13:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:04.752593 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac062f-6230-4f33-9f02-986f1a99593a None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:13:04.754155 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e7ac062f-6230-4f33-9f02-986f1a99593a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:04.854499 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac062f-6230-4f33-9f02-986f1a99593a None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:13:04.926031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f52bce-df87-4771-957d-eb7b4bbcb297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:13:04.926633 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f52bce-df87-4771-957d-eb7b4bbcb297 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:04.926904 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f52bce-df87-4771-957d-eb7b4bbcb297 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:04.927169 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f52bce-df87-4771-957d-eb7b4bbcb297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:04.928772 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-52f52bce-df87-4771-957d-eb7b4bbcb297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:04.964240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:05.041440 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f52bce-df87-4771-957d-eb7b4bbcb297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:13:05.043372 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f52bce-df87-4771-957d-eb7b4bbcb297 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:13:05.068044 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 507/1015] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:13:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:13:09.219932 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a96d0-d1bf-4852-81f1-a944b78ca86e None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:09.220465 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a96d0-d1bf-4852-81f1-a944b78ca86e None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:09.220465 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a96d0-d1bf-4852-81f1-a944b78ca86e None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:09.222571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 509/1016] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:13:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:09.227637 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e854053-e6ea-4da8-afc5-e07d5a242697 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:09.228049 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e854053-e6ea-4da8-afc5-e07d5a242697 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:09.228309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e854053-e6ea-4da8-afc5-e07d5a242697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:09.294015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-1e854053-e6ea-4da8-afc5-e07d5a242697 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:09.295257 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-1e854053-e6ea-4da8-afc5-e07d5a242697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:09.385185 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:09.397749 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 508/1017] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:09.691845 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a96d0-d1bf-4852-81f1-a944b78ca86e None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:13:09.693434 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-104a96d0-d1bf-4852-81f1-a944b78ca86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:09.744630 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:09.801313 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a96d0-d1bf-4852-81f1-a944b78ca86e None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:13:09.875672 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d1dc4-4b57-435b-9fa9-305ba405b28a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:13:09.876424 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d1dc4-4b57-435b-9fa9-305ba405b28a service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:09.876709 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d1dc4-4b57-435b-9fa9-305ba405b28a service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:09.876983 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d1dc4-4b57-435b-9fa9-305ba405b28a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:09.879130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-563d1dc4-4b57-435b-9fa9-305ba405b28a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:09.972612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:09.989341 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d1dc4-4b57-435b-9fa9-305ba405b28a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:13:09.990780 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d1dc4-4b57-435b-9fa9-305ba405b28a service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:13:10.013697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 510/1018] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:13:12.969832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df947a44-32c3-4ad3-a95f-48517b2bebec None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:12.970345 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df947a44-32c3-4ad3-a95f-48517b2bebec None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:12.970460 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df947a44-32c3-4ad3-a95f-48517b2bebec None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:12.972530 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 509/1019] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:13:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:12.978939 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c030d-1070-4b3d-bc69-8c68eca9c614 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:12.979398 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c030d-1070-4b3d-bc69-8c68eca9c614 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:12.979523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c030d-1070-4b3d-bc69-8c68eca9c614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:13.054610 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-631c030d-1070-4b3d-bc69-8c68eca9c614 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:13.055786 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-631c030d-1070-4b3d-bc69-8c68eca9c614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:13.156956 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 511/1020] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:13:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:13.457481 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29c860-9483-4f14-a86a-31ca0b409ad8 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:13.458021 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29c860-9483-4f14-a86a-31ca0b409ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:13.458351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29c860-9483-4f14-a86a-31ca0b409ad8 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:13.459997 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 510/1021] 10.222.0.13 () {54 vars in 964 bytes} [Fri Oct 21 21:13:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:13.466726 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eeb0ba-ee75-44cb-803d-d6c096cf366d None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:13.467281 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eeb0ba-ee75-44cb-803d-d6c096cf366d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:13.467557 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eeb0ba-ee75-44cb-803d-d6c096cf366d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:13.544633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:13.553497 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-e0eeb0ba-ee75-44cb-803d-d6c096cf366d None None] MFA Rules not processed for user `d34dc4e7c5474b6b96977019328d8890`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:13.554735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-e0eeb0ba-ee75-44cb-803d-d6c096cf366d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:13.632716 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 512/1022] 10.222.0.13 () {58 vars in 1061 bytes} [Fri Oct 21 21:13:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:13.642075 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29c860-9483-4f14-a86a-31ca0b409ad8 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:13:13.644188 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-fd29c860-9483-4f14-a86a-31ca0b409ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:13.749439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29c860-9483-4f14-a86a-31ca0b409ad8 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:13:13.846384 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a0972-782a-40b8-9e64-08a8b4848d4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:13:13.846996 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a0972-782a-40b8-9e64-08a8b4848d4d service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:13.847270 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a0972-782a-40b8-9e64-08a8b4848d4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:13.847541 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a0972-782a-40b8-9e64-08a8b4848d4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:13.849209 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-548a0972-782a-40b8-9e64-08a8b4848d4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:13.902494 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:13.972872 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a0972-782a-40b8-9e64-08a8b4848d4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:13:13.974409 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a0972-782a-40b8-9e64-08a8b4848d4d service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:13:13.994138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 511/1023] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:13:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:13:18.942946 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c979f-0787-4024-841a-fe5273a2e618 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:18.943483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c979f-0787-4024-841a-fe5273a2e618 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:18.943596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c979f-0787-4024-841a-fe5273a2e618 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:18.944904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 513/1024] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:13:18.950376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621331a9-6ed9-435f-96b3-37e0d29eb78c None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:18.950805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621331a9-6ed9-435f-96b3-37e0d29eb78c None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:18.950914 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621331a9-6ed9-435f-96b3-37e0d29eb78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:19.021411 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-621331a9-6ed9-435f-96b3-37e0d29eb78c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:19.022723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-621331a9-6ed9-435f-96b3-37e0d29eb78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:19.121352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:19.124665 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 512/1025] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:19.411365 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c979f-0787-4024-841a-fe5273a2e618 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:13:19.412929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-262c979f-0787-4024-841a-fe5273a2e618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:19.522566 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c979f-0787-4024-841a-fe5273a2e618 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:13:19.569422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:19.596272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8bfaf-dc80-4f59-b56e-42106866970f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:13:19.596877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8bfaf-dc80-4f59-b56e-42106866970f service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:19.597153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8bfaf-dc80-4f59-b56e-42106866970f service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:19.597419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8bfaf-dc80-4f59-b56e-42106866970f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:19.599002 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-09c8bfaf-dc80-4f59-b56e-42106866970f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:19.706591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8bfaf-dc80-4f59-b56e-42106866970f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:13:19.708049 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8bfaf-dc80-4f59-b56e-42106866970f service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:13:19.725874 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 514/1026] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:13:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:13:24.662459 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8bc88-d23d-4359-8a0d-7efac209213d None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:24.663421 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8bc88-d23d-4359-8a0d-7efac209213d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:24.663973 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8bc88-d23d-4359-8a0d-7efac209213d None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:24.665733 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 513/1027] 10.222.0.13 () {54 vars in 921 bytes} [Fri Oct 21 21:13:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:24.675364 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb0ef1-baf6-4fff-9b07-e7c927829d47 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:24.675978 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb0ef1-baf6-4fff-9b07-e7c927829d47 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:24.676656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb0ef1-baf6-4fff-9b07-e7c927829d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:24.818346 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-d7fb0ef1-baf6-4fff-9b07-e7c927829d47 None None] MFA Rules not processed for user `124f2b5b95e8450ea6867f6868b9e810`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:24.819828 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d7fb0ef1-baf6-4fff-9b07-e7c927829d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:24.841623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:24.963505 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 515/1028] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Oct 21 21:13:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:29.150971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d092086-fbd6-4be3-b810-61159674f729 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:29.151524 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d092086-fbd6-4be3-b810-61159674f729 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:29.151801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d092086-fbd6-4be3-b810-61159674f729 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:29.153211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 514/1029] 10.222.0.13 () {54 vars in 921 bytes} [Fri Oct 21 21:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:29.159335 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c635fc3-b40d-4a63-8879-bc488e9bb7f5 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:29.159845 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c635fc3-b40d-4a63-8879-bc488e9bb7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:29.160090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c635fc3-b40d-4a63-8879-bc488e9bb7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:29.242387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-0c635fc3-b40d-4a63-8879-bc488e9bb7f5 None None] MFA Rules not processed for user `124f2b5b95e8450ea6867f6868b9e810`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:29.243663 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-0c635fc3-b40d-4a63-8879-bc488e9bb7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:29.320586 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:29.360367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 516/1030] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Oct 21 21:13:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:13:30.764790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f79d9-2783-46f3-8dbb-d545fddd1552 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:30.765324 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f79d9-2783-46f3-8dbb-d545fddd1552 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:30.765324 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f79d9-2783-46f3-8dbb-d545fddd1552 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:30.766716 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 515/1031] 10.222.0.13 () {54 vars in 921 bytes} [Fri Oct 21 21:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:30.772813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466cfce-ec3e-490a-9a67-8c5edba16805 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:30.773238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466cfce-ec3e-490a-9a67-8c5edba16805 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:30.773381 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466cfce-ec3e-490a-9a67-8c5edba16805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:30.850986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-1466cfce-ec3e-490a-9a67-8c5edba16805 None None] MFA Rules not processed for user `124f2b5b95e8450ea6867f6868b9e810`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:30.852240 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1466cfce-ec3e-490a-9a67-8c5edba16805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:30.969016 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 517/1032] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Oct 21 21:13:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:34.430510 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254920b-c343-4346-a66b-f271552001da None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:34.431052 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254920b-c343-4346-a66b-f271552001da None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:34.431052 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254920b-c343-4346-a66b-f271552001da None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:34.432285 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 516/1033] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:34.437758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b93dc-dbc9-4293-ba6b-81400ff03837 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:34.438262 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b93dc-dbc9-4293-ba6b-81400ff03837 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:34.438796 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b93dc-dbc9-4293-ba6b-81400ff03837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:34.481540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:34.517250 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-733b93dc-dbc9-4293-ba6b-81400ff03837 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:34.518486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-733b93dc-dbc9-4293-ba6b-81400ff03837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:34.633221 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 518/1034] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:34.949734 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a70dbb-d9f7-48cd-a5ba-768c897e49de None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:34.950208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a70dbb-d9f7-48cd-a5ba-768c897e49de None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:34.950617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a70dbb-d9f7-48cd-a5ba-768c897e49de None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:34.952390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 517/1035] 10.222.0.13 () {54 vars in 961 bytes} [Fri Oct 21 21:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:34.977755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38cada-7303-48a9-8fd0-004704252a88 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:34.978172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38cada-7303-48a9-8fd0-004704252a88 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:34.978655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38cada-7303-48a9-8fd0-004704252a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:35.069790 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-7d38cada-7303-48a9-8fd0-004704252a88 None None] MFA Rules not processed for user `12dad9f016a740b98538722e1b723823`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:35.071186 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-7d38cada-7303-48a9-8fd0-004704252a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:35.125835 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:35.202669 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 519/1036] 10.222.0.13 () {58 vars in 1058 bytes} [Fri Oct 21 21:13:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:35.210941 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a70dbb-d9f7-48cd-a5ba-768c897e49de None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:13:35.212879 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-80a70dbb-d9f7-48cd-a5ba-768c897e49de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:35.367393 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a70dbb-d9f7-48cd-a5ba-768c897e49de None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:13:35.444728 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:35.495333 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680df34-0786-4073-9da0-78b9550a4cbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:13:35.496028 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680df34-0786-4073-9da0-78b9550a4cbf service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:35.496322 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680df34-0786-4073-9da0-78b9550a4cbf service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:35.496603 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680df34-0786-4073-9da0-78b9550a4cbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:35.499365 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-7680df34-0786-4073-9da0-78b9550a4cbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:35.602686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680df34-0786-4073-9da0-78b9550a4cbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:13:35.604333 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680df34-0786-4073-9da0-78b9550a4cbf service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:13:35.625650 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 518/1037] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:13:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:13:38.095718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a9a3a-6d23-4249-ad76-ad242ca71704 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:38.096278 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a9a3a-6d23-4249-ad76-ad242ca71704 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:38.096376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a9a3a-6d23-4249-ad76-ad242ca71704 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:38.097943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 520/1038] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:13:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:38.106148 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92ab50-7765-49d9-8acd-f39d5200a584 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:38.106803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92ab50-7765-49d9-8acd-f39d5200a584 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:38.107473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92ab50-7765-49d9-8acd-f39d5200a584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:38.183356 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-5d92ab50-7765-49d9-8acd-f39d5200a584 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:38.184663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-5d92ab50-7765-49d9-8acd-f39d5200a584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:38.225117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:38.298324 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 519/1039] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:38.601984 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a9a3a-6d23-4249-ad76-ad242ca71704 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:13:38.603838 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d53a9a3a-6d23-4249-ad76-ad242ca71704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:38.782813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a9a3a-6d23-4249-ad76-ad242ca71704 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:13:38.792905 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:38.898598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648233a-9b85-4e86-8370-a82843ea571e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:13:38.899206 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648233a-9b85-4e86-8370-a82843ea571e service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:38.899478 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648233a-9b85-4e86-8370-a82843ea571e service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:38.899775 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648233a-9b85-4e86-8370-a82843ea571e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:38.901838 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8648233a-9b85-4e86-8370-a82843ea571e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:38.998672 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648233a-9b85-4e86-8370-a82843ea571e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:13:39.000967 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648233a-9b85-4e86-8370-a82843ea571e service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:13:39.020097 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 521/1040] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:13:41.546152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b0db3-f233-4013-8b54-f9ce37340fe3 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:41.546723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b0db3-f233-4013-8b54-f9ce37340fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:41.546723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b0db3-f233-4013-8b54-f9ce37340fe3 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:41.548800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 520/1041] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:13:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:41.553906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c036aa-ee03-49ac-bdf7-c6f0d586bce5 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:41.554313 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c036aa-ee03-49ac-bdf7-c6f0d586bce5 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:41.554458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c036aa-ee03-49ac-bdf7-c6f0d586bce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:41.570663 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:41.629559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-c7c036aa-ee03-49ac-bdf7-c6f0d586bce5 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:41.631291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c7c036aa-ee03-49ac-bdf7-c6f0d586bce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:41.754543 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 522/1042] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:42.056099 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fb864-7462-4e14-8839-57ebb3fd03a1 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:42.056519 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fb864-7462-4e14-8839-57ebb3fd03a1 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:42.056915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fb864-7462-4e14-8839-57ebb3fd03a1 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:42.058383 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 521/1043] 10.222.0.13 () {54 vars in 961 bytes} [Fri Oct 21 21:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:42.064892 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa7a72-0315-4a7b-8d2e-2030a07bc0c1 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:42.065286 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa7a72-0315-4a7b-8d2e-2030a07bc0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:42.065658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa7a72-0315-4a7b-8d2e-2030a07bc0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:42.149763 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-07fa7a72-0315-4a7b-8d2e-2030a07bc0c1 None None] MFA Rules not processed for user `12dad9f016a740b98538722e1b723823`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:42.151143 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-07fa7a72-0315-4a7b-8d2e-2030a07bc0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:42.190319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:42.280129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 523/1044] 10.222.0.13 () {58 vars in 1058 bytes} [Fri Oct 21 21:13:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:42.288390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7fb864-7462-4e14-8839-57ebb3fd03a1 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:13:42.292432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0e7fb864-7462-4e14-8839-57ebb3fd03a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:42.450245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7fb864-7462-4e14-8839-57ebb3fd03a1 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:13:42.473172 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:42.580619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db502399-4be3-4531-8bfd-44c7c51cc563 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:13:42.581365 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db502399-4be3-4531-8bfd-44c7c51cc563 service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:42.581475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db502399-4be3-4531-8bfd-44c7c51cc563 service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:42.581669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db502399-4be3-4531-8bfd-44c7c51cc563 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:42.583828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-db502399-4be3-4531-8bfd-44c7c51cc563 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:42.680954 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92bead-994c-42bf-bbe6-c6605ef82e04 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:42.681493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92bead-994c-42bf-bbe6-c6605ef82e04 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:42.681493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92bead-994c-42bf-bbe6-c6605ef82e04 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:42.683030 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 524/1045] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:42.686994 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db502399-4be3-4531-8bfd-44c7c51cc563 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:13:42.689316 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db502399-4be3-4531-8bfd-44c7c51cc563 service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:13:42.689486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1f562-bb1e-4c6d-9a89-aaefee2fc4c2 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:42.689776 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1f562-bb1e-4c6d-9a89-aaefee2fc4c2 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:42.690052 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1f562-bb1e-4c6d-9a89-aaefee2fc4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:42.712357 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 522/1046] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:13:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:13:42.778606 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-aae1f562-bb1e-4c6d-9a89-aaefee2fc4c2 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:42.779972 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-aae1f562-bb1e-4c6d-9a89-aaefee2fc4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:42.900588 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:42.915554 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 525/1047] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:43.219464 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d8367-7d2d-47fa-9a55-f5ed26f51cfa None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:43.219813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d8367-7d2d-47fa-9a55-f5ed26f51cfa None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:43.220078 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d8367-7d2d-47fa-9a55-f5ed26f51cfa None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:43.221454 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 523/1048] 10.222.0.13 () {54 vars in 963 bytes} [Fri Oct 21 21:13:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:43.227095 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11436085-e0dc-4da6-9b93-3986cd5203c1 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:43.227386 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11436085-e0dc-4da6-9b93-3986cd5203c1 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:43.227704 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11436085-e0dc-4da6-9b93-3986cd5203c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:43.314016 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-11436085-e0dc-4da6-9b93-3986cd5203c1 None None] MFA Rules not processed for user `d620cad0ef21453a9cab61f728913cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:43.315327 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-11436085-e0dc-4da6-9b93-3986cd5203c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:43.399900 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 526/1049] 10.222.0.13 () {58 vars in 1060 bytes} [Fri Oct 21 21:13:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:43.404454 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d8367-7d2d-47fa-9a55-f5ed26f51cfa None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:13:43.406028 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-db8d8367-7d2d-47fa-9a55-f5ed26f51cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:43.435366 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:43.526319 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d8367-7d2d-47fa-9a55-f5ed26f51cfa None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:13:43.614941 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7ca15-ec49-4b2c-9b8d-4b4c3c88f0b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:13:43.615588 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7ca15-ec49-4b2c-9b8d-4b4c3c88f0b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:43.616012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7ca15-ec49-4b2c-9b8d-4b4c3c88f0b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:43.616161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7ca15-ec49-4b2c-9b8d-4b4c3c88f0b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:43.617772 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b3a7ca15-ec49-4b2c-9b8d-4b4c3c88f0b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:43.712805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:43.761271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7ca15-ec49-4b2c-9b8d-4b4c3c88f0b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:13:43.762704 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7ca15-ec49-4b2c-9b8d-4b4c3c88f0b0 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:13:43.784117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 524/1050] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:13:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:13:45.022777 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a5502-98e9-46bb-ab9c-e12f4ccef8b1 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:45.023298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a5502-98e9-46bb-ab9c-e12f4ccef8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:45.023298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a5502-98e9-46bb-ab9c-e12f4ccef8b1 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:45.024664 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 527/1051] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:45.029987 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443e55a-0b2d-4e7e-b132-ca71f139ebf9 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:45.030437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443e55a-0b2d-4e7e-b132-ca71f139ebf9 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:45.030529 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443e55a-0b2d-4e7e-b132-ca71f139ebf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:45.097549 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-f443e55a-0b2d-4e7e-b132-ca71f139ebf9 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:45.098774 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f443e55a-0b2d-4e7e-b132-ca71f139ebf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:45.198092 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 525/1052] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:45.499185 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a5502-98e9-46bb-ab9c-e12f4ccef8b1 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:13:45.500821 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-471a5502-98e9-46bb-ab9c-e12f4ccef8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:45.589435 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:45.656949 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a5502-98e9-46bb-ab9c-e12f4ccef8b1 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:13:45.790818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42aa18-f89a-47a3-ace2-91a0586bfc38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:13:45.791439 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42aa18-f89a-47a3-ace2-91a0586bfc38 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:45.791781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42aa18-f89a-47a3-ace2-91a0586bfc38 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:45.792049 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42aa18-f89a-47a3-ace2-91a0586bfc38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:45.793626 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ed42aa18-f89a-47a3-ace2-91a0586bfc38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:45.845745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:45.891665 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42aa18-f89a-47a3-ace2-91a0586bfc38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:13:45.893308 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42aa18-f89a-47a3-ace2-91a0586bfc38 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:13:45.913298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 528/1053] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:13:48.201090 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b92d9-7eda-44b6-bec0-558e76c79566 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:48.201612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b92d9-7eda-44b6-bec0-558e76c79566 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:48.201907 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b92d9-7eda-44b6-bec0-558e76c79566 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:48.203530 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 526/1054] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:48.208723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d5431-4c71-4693-9ff4-e5fed0a7f1c4 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:48.209123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d5431-4c71-4693-9ff4-e5fed0a7f1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:48.209287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d5431-4c71-4693-9ff4-e5fed0a7f1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:48.279056 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-4c4d5431-4c71-4693-9ff4-e5fed0a7f1c4 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:48.280424 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-4c4d5431-4c71-4693-9ff4-e5fed0a7f1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:48.383525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 529/1055] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:13:48.682553 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b92d9-7eda-44b6-bec0-558e76c79566 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:13:48.684080 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-fe6b92d9-7eda-44b6-bec0-558e76c79566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:48.731134 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:48.852332 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b92d9-7eda-44b6-bec0-558e76c79566 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:13:48.972547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:48.978897 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb621a2e-1974-4153-8eab-5a07adef92d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:13:48.979505 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb621a2e-1974-4153-8eab-5a07adef92d7 service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:48.979808 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb621a2e-1974-4153-8eab-5a07adef92d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:48.980084 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb621a2e-1974-4153-8eab-5a07adef92d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:48.982103 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-bb621a2e-1974-4153-8eab-5a07adef92d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:49.080364 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb621a2e-1974-4153-8eab-5a07adef92d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:13:49.081964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb621a2e-1974-4153-8eab-5a07adef92d7 service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:13:49.100552 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 527/1056] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:13:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:13:51.322697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7db9a0-b1e3-4212-ac2c-549c5f41b234 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:51.323184 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7db9a0-b1e3-4212-ac2c-549c5f41b234 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:51.323184 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7db9a0-b1e3-4212-ac2c-549c5f41b234 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:51.324552 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 530/1057] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:51.330758 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c4b49-932c-4fb0-bdb6-185bad0d1bf2 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:51.331203 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c4b49-932c-4fb0-bdb6-185bad0d1bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:51.331332 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c4b49-932c-4fb0-bdb6-185bad0d1bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:51.398708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-ac5c4b49-932c-4fb0-bdb6-185bad0d1bf2 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:51.400088 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ac5c4b49-932c-4fb0-bdb6-185bad0d1bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:51.465992 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:51.499144 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 528/1058] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:51.795238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7db9a0-b1e3-4212-ac2c-549c5f41b234 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:13:51.796793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-0a7db9a0-b1e3-4212-ac2c-549c5f41b234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:51.826261 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:51.942100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7db9a0-b1e3-4212-ac2c-549c5f41b234 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:13:52.046282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:52.064199 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b2b74-e34e-4024-8bde-60351d1c51b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:13:52.064804 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b2b74-e34e-4024-8bde-60351d1c51b8 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:52.065092 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b2b74-e34e-4024-8bde-60351d1c51b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:52.065345 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b2b74-e34e-4024-8bde-60351d1c51b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:52.066906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-663b2b74-e34e-4024-8bde-60351d1c51b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:52.155676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b2b74-e34e-4024-8bde-60351d1c51b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:13:52.157262 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b2b74-e34e-4024-8bde-60351d1c51b8 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:13:52.175159 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 531/1059] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:13:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:13:54.786212 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3969b1-c6c2-4c01-9bfe-d9c824026135 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:54.786956 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3969b1-c6c2-4c01-9bfe-d9c824026135 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:54.786956 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3969b1-c6c2-4c01-9bfe-d9c824026135 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:54.788223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 529/1060] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:13:54.794299 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac5561-9011-402c-86fb-49fc4d5aefa1 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:54.794822 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac5561-9011-402c-86fb-49fc4d5aefa1 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:54.795020 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac5561-9011-402c-86fb-49fc4d5aefa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:54.877681 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-d9ac5561-9011-402c-86fb-49fc4d5aefa1 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:54.879244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d9ac5561-9011-402c-86fb-49fc4d5aefa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:54.944511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:54.986689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 532/1061] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:55.275856 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3969b1-c6c2-4c01-9bfe-d9c824026135 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:13:55.277409 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-4b3969b1-c6c2-4c01-9bfe-d9c824026135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:55.432134 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3969b1-c6c2-4c01-9bfe-d9c824026135 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:13:55.493808 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:55.557303 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0bf82-de66-434c-a41a-240c3ed2431b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:13:55.557831 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0bf82-de66-434c-a41a-240c3ed2431b service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:55.558106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0bf82-de66-434c-a41a-240c3ed2431b service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:55.558375 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0bf82-de66-434c-a41a-240c3ed2431b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:55.559990 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-3cb0bf82-de66-434c-a41a-240c3ed2431b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:55.657261 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0bf82-de66-434c-a41a-240c3ed2431b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:13:55.659592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0bf82-de66-434c-a41a-240c3ed2431b service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:13:55.681439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 530/1062] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:13:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:13:58.192574 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece7cd2-d62f-41b4-86b4-6fb7c1c2e57f None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:58.193219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece7cd2-d62f-41b4-86b4-6fb7c1c2e57f None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:58.193219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece7cd2-d62f-41b4-86b4-6fb7c1c2e57f None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:58.194430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 533/1063] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:13:58.200011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae161e-b3ef-4e33-b950-028530f2d901 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:58.200508 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae161e-b3ef-4e33-b950-028530f2d901 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:58.200689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae161e-b3ef-4e33-b950-028530f2d901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:58.267063 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-71ae161e-b3ef-4e33-b950-028530f2d901 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:13:58.268296 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-71ae161e-b3ef-4e33-b950-028530f2d901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:58.277366 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:58.368799 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 531/1064] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:13:58.654026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece7cd2-d62f-41b4-86b4-6fb7c1c2e57f None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:13:58.656135 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-0ece7cd2-d62f-41b4-86b4-6fb7c1c2e57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:58.803354 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece7cd2-d62f-41b4-86b4-6fb7c1c2e57f None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:13:58.848724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:13:58.917342 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffb26b-23bb-4ab5-aee1-991498ad48fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:13:58.917825 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffb26b-23bb-4ab5-aee1-991498ad48fa service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:13:58.918100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffb26b-23bb-4ab5-aee1-991498ad48fa service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:13:58.918369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffb26b-23bb-4ab5-aee1-991498ad48fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:13:58.919966 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3fffb26b-23bb-4ab5-aee1-991498ad48fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:13:59.008023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffb26b-23bb-4ab5-aee1-991498ad48fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:13:59.009592 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffb26b-23bb-4ab5-aee1-991498ad48fa service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:13:59.027662 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 534/1065] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:13:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:00.315956 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febe609-bc82-405f-9cb3-3522de8df7c1 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:00.316451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febe609-bc82-405f-9cb3-3522de8df7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:00.316558 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febe609-bc82-405f-9cb3-3522de8df7c1 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:00.317840 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 532/1066] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:00.323265 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ee123-05a0-4295-9498-7cca55061f46 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:00.323720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ee123-05a0-4295-9498-7cca55061f46 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:00.323816 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ee123-05a0-4295-9498-7cca55061f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:00.386961 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:00.390020 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-5e4ee123-05a0-4295-9498-7cca55061f46 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:00.391763 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-5e4ee123-05a0-4295-9498-7cca55061f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:00.488660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 535/1067] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:00.789564 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dc737-66d6-4743-9274-01eac541d17a None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:00.789849 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dc737-66d6-4743-9274-01eac541d17a None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:00.790123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dc737-66d6-4743-9274-01eac541d17a None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:00.791648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 533/1068] 10.222.0.13 () {54 vars in 962 bytes} [Fri Oct 21 21:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:00.796660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa276fc-5524-446d-a8b4-e4ebe0c28af9 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:00.796946 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa276fc-5524-446d-a8b4-e4ebe0c28af9 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:00.797217 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa276fc-5524-446d-a8b4-e4ebe0c28af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:00.872104 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-afa276fc-5524-446d-a8b4-e4ebe0c28af9 None None] MFA Rules not processed for user `5ce5ec441b06408bbadd6fc040af0da5`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:00.873292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-afa276fc-5524-446d-a8b4-e4ebe0c28af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:00.944966 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:00.951496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 536/1069] 10.222.0.13 () {58 vars in 1059 bytes} [Fri Oct 21 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:14:00.960127 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dc737-66d6-4743-9274-01eac541d17a None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:00.961645 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-217dc737-66d6-4743-9274-01eac541d17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:01.063755 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dc737-66d6-4743-9274-01eac541d17a None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:01.104424 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:01.144043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54d46e-5c33-48ba-8d2f-f707c017c26d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce5ec441b06408bbadd6fc040af0da5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:01.144647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54d46e-5c33-48ba-8d2f-f707c017c26d service glance] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:01.144918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54d46e-5c33-48ba-8d2f-f707c017c26d service glance] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:01.145184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54d46e-5c33-48ba-8d2f-f707c017c26d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:01.146811 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-db54d46e-5c33-48ba-8d2f-f707c017c26d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:01.207788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d251a9-cbd2-4751-b64b-a51afe0008b7 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:01.208075 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d251a9-cbd2-4751-b64b-a51afe0008b7 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:01.208350 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d251a9-cbd2-4751-b64b-a51afe0008b7 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:01.209758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 537/1070] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:01.214919 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a2460-0262-40cc-a861-8feefe51c0c4 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:01.215261 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a2460-0262-40cc-a861-8feefe51c0c4 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:01.215621 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a2460-0262-40cc-a861-8feefe51c0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:01.255167 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54d46e-5c33-48ba-8d2f-f707c017c26d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:01.256645 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54d46e-5c33-48ba-8d2f-f707c017c26d service glance] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:01.274954 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 534/1071] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:01.283900 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-633a2460-0262-40cc-a861-8feefe51c0c4 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:01.285177 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-633a2460-0262-40cc-a861-8feefe51c0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:01.380402 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 538/1072] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:14:01.675524 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54d46e-5c33-48ba-8d2f-f707c017c26d service glance] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:01.676762 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-db54d46e-5c33-48ba-8d2f-f707c017c26d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:01.755251 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:01.791897 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54d46e-5c33-48ba-8d2f-f707c017c26d service glance] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:01.922535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443d2e2-a05b-4820-906a-d2b00b16104e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:01.923038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443d2e2-a05b-4820-906a-d2b00b16104e service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:01.923315 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443d2e2-a05b-4820-906a-d2b00b16104e service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:01.923611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443d2e2-a05b-4820-906a-d2b00b16104e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:01.925256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f443d2e2-a05b-4820-906a-d2b00b16104e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:02.006502 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:02.020650 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443d2e2-a05b-4820-906a-d2b00b16104e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:02.022428 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443d2e2-a05b-4820-906a-d2b00b16104e service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:02.041284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 535/1073] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:02.042419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc10b68-811e-45f3-bca2-10cbd9e3c4c1 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:02.042711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc10b68-811e-45f3-bca2-10cbd9e3c4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:02.042992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc10b68-811e-45f3-bca2-10cbd9e3c4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:02.112330 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-2dc10b68-811e-45f3-bca2-10cbd9e3c4c1 None None] MFA Rules not processed for user `2346003afe74477496045e9bc72173d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:02.113559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2dc10b68-811e-45f3-bca2-10cbd9e3c4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:02.144296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:02.210907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 539/1074] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Oct 21 21:14:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:14:02.256264 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7353353-870e-45d4-9918-1eeb481b5355 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:02.256546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7353353-870e-45d4-9918-1eeb481b5355 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:02.256818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7353353-870e-45d4-9918-1eeb481b5355 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:02.258199 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 536/1075] 10.222.0.13 () {54 vars in 943 bytes} [Fri Oct 21 21:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:02.265587 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f7f7b-0db2-4f37-97ac-caeb30524f8b None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:02.265875 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f7f7b-0db2-4f37-97ac-caeb30524f8b None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:02.266148 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f7f7b-0db2-4f37-97ac-caeb30524f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:02.341812 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-919f7f7b-0db2-4f37-97ac-caeb30524f8b None None] MFA Rules not processed for user `7ce720c8df014f9a8af062138aecd1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:02.343037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-919f7f7b-0db2-4f37-97ac-caeb30524f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:02.446447 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:02.451770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 540/1076] 10.222.0.13 () {58 vars in 1040 bytes} [Fri Oct 21 21:14:02 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:14:02.460008 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7353353-870e-45d4-9918-1eeb481b5355 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:02.462412 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-a7353353-870e-45d4-9918-1eeb481b5355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:02.596509 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7353353-870e-45d4-9918-1eeb481b5355 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:02.660300 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:02.699830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff7605-9852-4d74-9dad-e38e6e6862e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ce720c8df014f9a8af062138aecd1b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:02.700455 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff7605-9852-4d74-9dad-e38e6e6862e5 service swift] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:02.700716 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff7605-9852-4d74-9dad-e38e6e6862e5 service swift] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:02.700985 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff7605-9852-4d74-9dad-e38e6e6862e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:02.703121 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-29ff7605-9852-4d74-9dad-e38e6e6862e5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:02.789246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff7605-9852-4d74-9dad-e38e6e6862e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:02.790833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff7605-9852-4d74-9dad-e38e6e6862e5 service swift] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:02.799531 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 537/1077] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Oct 21 21:14:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 21:14:04.273723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e075ded-c9bc-41c3-899e-f2f8c3af7a60 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:04.274377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e075ded-c9bc-41c3-899e-f2f8c3af7a60 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:04.274377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e075ded-c9bc-41c3-899e-f2f8c3af7a60 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:04.275651 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 541/1078] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:04.281051 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842eeb13-7255-4b2f-8992-e179728157a8 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:04.281597 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842eeb13-7255-4b2f-8992-e179728157a8 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:04.281714 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842eeb13-7255-4b2f-8992-e179728157a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:04.354751 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-842eeb13-7255-4b2f-8992-e179728157a8 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:04.355956 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-842eeb13-7255-4b2f-8992-e179728157a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:04.418522 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:04.467801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 538/1079] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:04.763461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e075ded-c9bc-41c3-899e-f2f8c3af7a60 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:04.765097 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9e075ded-c9bc-41c3-899e-f2f8c3af7a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:04.929042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e075ded-c9bc-41c3-899e-f2f8c3af7a60 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:05.036012 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:05.091388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ef108-c489-401e-b7fa-0096f7acfd97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:05.092054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ef108-c489-401e-b7fa-0096f7acfd97 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:05.092329 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ef108-c489-401e-b7fa-0096f7acfd97 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:05.092653 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ef108-c489-401e-b7fa-0096f7acfd97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:05.094313 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-b58ef108-c489-401e-b7fa-0096f7acfd97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:05.193599 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ef108-c489-401e-b7fa-0096f7acfd97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:05.195677 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ef108-c489-401e-b7fa-0096f7acfd97 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:05.217000 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 542/1080] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:07.459915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577ae40-3b52-4055-8bb4-d943c9cad634 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:07.460477 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577ae40-3b52-4055-8bb4-d943c9cad634 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:07.460477 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577ae40-3b52-4055-8bb4-d943c9cad634 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:07.463041 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 539/1081] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:07.467971 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b76e9-8832-400f-b029-815aa79d798a None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:07.468481 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b76e9-8832-400f-b029-815aa79d798a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:07.468733 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b76e9-8832-400f-b029-815aa79d798a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:07.539545 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-3d1b76e9-8832-400f-b029-815aa79d798a None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:07.540819 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3d1b76e9-8832-400f-b029-815aa79d798a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:07.586515 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:07.656090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 543/1082] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:07.956358 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577ae40-3b52-4055-8bb4-d943c9cad634 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:07.957908 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b577ae40-3b52-4055-8bb4-d943c9cad634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:08.151616 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577ae40-3b52-4055-8bb4-d943c9cad634 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:08.203098 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:08.312035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179810d-5267-4979-a88a-ca76f6c6060f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:08.312655 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179810d-5267-4979-a88a-ca76f6c6060f service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:08.312948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179810d-5267-4979-a88a-ca76f6c6060f service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:08.313948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179810d-5267-4979-a88a-ca76f6c6060f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:08.315641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-4179810d-5267-4979-a88a-ca76f6c6060f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:08.416430 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4179810d-5267-4979-a88a-ca76f6c6060f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:08.418281 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4179810d-5267-4979-a88a-ca76f6c6060f service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:08.441830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 540/1083] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:10.725827 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065df8-9402-4eb8-bdb0-f44e4e46834d None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:10.726391 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065df8-9402-4eb8-bdb0-f44e4e46834d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:10.726484 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065df8-9402-4eb8-bdb0-f44e4e46834d None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:10.734510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 544/1084] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:10.736170 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a49e2-3578-404c-bc79-62f2197bea42 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:10.736613 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a49e2-3578-404c-bc79-62f2197bea42 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:10.737158 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a49e2-3578-404c-bc79-62f2197bea42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:10.791409 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:10.810656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-331a49e2-3578-404c-bc79-62f2197bea42 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:10.812508 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-331a49e2-3578-404c-bc79-62f2197bea42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:10.921251 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 541/1085] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:11.230682 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e065df8-9402-4eb8-bdb0-f44e4e46834d None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:11.232356 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2e065df8-9402-4eb8-bdb0-f44e4e46834d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:11.393236 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e065df8-9402-4eb8-bdb0-f44e4e46834d None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:11.417267 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:11.528110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03014f3-2f47-47d6-86cb-27bd2e60153b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:11.528675 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03014f3-2f47-47d6-86cb-27bd2e60153b service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:11.528969 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03014f3-2f47-47d6-86cb-27bd2e60153b service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:11.529259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03014f3-2f47-47d6-86cb-27bd2e60153b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:11.531437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-e03014f3-2f47-47d6-86cb-27bd2e60153b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:11.629423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03014f3-2f47-47d6-86cb-27bd2e60153b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:11.631048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03014f3-2f47-47d6-86cb-27bd2e60153b service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:11.651013 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 545/1086] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:20.919068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=47 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:21.648214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=90 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:25.924170 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916cab5-10f2-4cfd-b623-a96caacafca9 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:25.924726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916cab5-10f2-4cfd-b623-a96caacafca9 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:25.924726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916cab5-10f2-4cfd-b623-a96caacafca9 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:25.926065 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 542/1087] 10.222.0.13 () {54 vars in 918 bytes} [Fri Oct 21 21:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:25.931292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d8497-cb50-42fb-97a2-208b5110fa1c None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:25.931775 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d8497-cb50-42fb-97a2-208b5110fa1c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:25.931878 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d8497-cb50-42fb-97a2-208b5110fa1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:26.007772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-6b7d8497-cb50-42fb-97a2-208b5110fa1c None None] MFA Rules not processed for user `124f2b5b95e8450ea6867f6868b9e810`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:26.008890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-6b7d8497-cb50-42fb-97a2-208b5110fa1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:26.112869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 546/1088] 10.222.0.13 () {58 vars in 1015 bytes} [Fri Oct 21 21:14:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:26.126938 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf6590-ba3e-4057-a3bd-e02ae0e7d12e None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:26.127221 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf6590-ba3e-4057-a3bd-e02ae0e7d12e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:26.127492 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf6590-ba3e-4057-a3bd-e02ae0e7d12e None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:26.129116 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 543/1089] 10.222.0.13 () {54 vars in 918 bytes} [Fri Oct 21 21:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:26.134358 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980fb64-8a16-4c0f-b3c7-1fe79fa8df8f None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:26.134646 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980fb64-8a16-4c0f-b3c7-1fe79fa8df8f None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:26.134921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980fb64-8a16-4c0f-b3c7-1fe79fa8df8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:26.208403 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:26.211725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-7980fb64-8a16-4c0f-b3c7-1fe79fa8df8f None None] MFA Rules not processed for user `12dad9f016a740b98538722e1b723823`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:26.213477 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-7980fb64-8a16-4c0f-b3c7-1fe79fa8df8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:26.326642 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 547/1090] 10.222.0.13 () {58 vars in 1015 bytes} [Fri Oct 21 21:14:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:28.895958 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1e3ee-6561-4809-9dfe-06d48772c6f7 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:28.896514 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1e3ee-6561-4809-9dfe-06d48772c6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:28.896514 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1e3ee-6561-4809-9dfe-06d48772c6f7 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:28.897827 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 544/1091] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 21:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:28.903500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7965f10-db63-4574-b93b-c5a107d2c596 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:28.904635 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7965f10-db63-4574-b93b-c5a107d2c596 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:28.904635 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7965f10-db63-4574-b93b-c5a107d2c596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:28.969465 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-f7965f10-db63-4574-b93b-c5a107d2c596 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:28.970658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-f7965f10-db63-4574-b93b-c5a107d2c596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:29.024639 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:29.066051 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 548/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 21:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:14:29.353788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1e3ee-6561-4809-9dfe-06d48772c6f7 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:29.355491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-c2b1e3ee-6561-4809-9dfe-06d48772c6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:29.497755 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1e3ee-6561-4809-9dfe-06d48772c6f7 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:29.567797 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:29.607965 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27962993-14ff-4704-ae90-886ce2370b15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:29.608511 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27962993-14ff-4704-ae90-886ce2370b15 service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:29.608768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27962993-14ff-4704-ae90-886ce2370b15 service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:29.609035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27962993-14ff-4704-ae90-886ce2370b15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:29.610609 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-27962993-14ff-4704-ae90-886ce2370b15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:29.695475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27962993-14ff-4704-ae90-886ce2370b15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:29.697164 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27962993-14ff-4704-ae90-886ce2370b15 service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:29.715140 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 545/1093] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:30.853700 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802b2d8-72d7-4a4a-ae07-cbbf6868db27 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:30.854260 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802b2d8-72d7-4a4a-ae07-cbbf6868db27 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:30.854260 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802b2d8-72d7-4a4a-ae07-cbbf6868db27 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:30.855640 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 549/1094] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 21:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:30.861825 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7550e7-bedc-465a-9977-e6a0e10d3862 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:30.862245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7550e7-bedc-465a-9977-e6a0e10d3862 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:30.862354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7550e7-bedc-465a-9977-e6a0e10d3862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:30.940867 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-de7550e7-bedc-465a-9977-e6a0e10d3862 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:30.942058 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-de7550e7-bedc-465a-9977-e6a0e10d3862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:30.968349 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:31.039481 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 546/1095] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 21:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:31.202752 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802b2d8-72d7-4a4a-ae07-cbbf6868db27 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:31.204306 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8802b2d8-72d7-4a4a-ae07-cbbf6868db27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:31.348700 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802b2d8-72d7-4a4a-ae07-cbbf6868db27 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:31.389077 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:31.460989 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed8c06-0986-4523-b45b-054e61aa8be1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:31.461598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed8c06-0986-4523-b45b-054e61aa8be1 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:31.461872 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed8c06-0986-4523-b45b-054e61aa8be1 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:31.462259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed8c06-0986-4523-b45b-054e61aa8be1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:31.464000 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-92ed8c06-0986-4523-b45b-054e61aa8be1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:31.551943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed8c06-0986-4523-b45b-054e61aa8be1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:31.553505 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed8c06-0986-4523-b45b-054e61aa8be1 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:31.572256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 550/1096] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:32.707117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e18bf-8028-4e84-9f0c-edfca43a009b None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:32.707702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e18bf-8028-4e84-9f0c-edfca43a009b None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:32.707702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e18bf-8028-4e84-9f0c-edfca43a009b None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:32.709036 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 547/1097] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 21:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:32.715201 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fb9c8-70c5-4e37-9f90-615c7b78b481 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:32.715686 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fb9c8-70c5-4e37-9f90-615c7b78b481 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:32.715814 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fb9c8-70c5-4e37-9f90-615c7b78b481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:32.771589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:32.794414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-c09fb9c8-70c5-4e37-9f90-615c7b78b481 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:32.795655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c09fb9c8-70c5-4e37-9f90-615c7b78b481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:32.890374 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 551/1098] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 21:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:33.040894 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e18bf-8028-4e84-9f0c-edfca43a009b None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:33.042357 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-230e18bf-8028-4e84-9f0c-edfca43a009b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:33.183488 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e18bf-8028-4e84-9f0c-edfca43a009b None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:33.195238 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:33.297373 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d656ca-9e17-4606-acc9-3dfca8217c8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:33.297898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d656ca-9e17-4606-acc9-3dfca8217c8f service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:33.298176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d656ca-9e17-4606-acc9-3dfca8217c8f service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:33.298445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d656ca-9e17-4606-acc9-3dfca8217c8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:33.300076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-40d656ca-9e17-4606-acc9-3dfca8217c8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:33.393402 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d656ca-9e17-4606-acc9-3dfca8217c8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:33.394989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d656ca-9e17-4606-acc9-3dfca8217c8f service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:33.412599 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 548/1099] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:34.522181 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90efbc8-6889-4e1d-8bf8-ec506d25b258 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:34.522750 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90efbc8-6889-4e1d-8bf8-ec506d25b258 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:34.522750 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90efbc8-6889-4e1d-8bf8-ec506d25b258 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:34.524104 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 552/1100] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 21:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:34.530910 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e89c26-9c04-4c61-ac6a-aca521cdaec2 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:34.531343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e89c26-9c04-4c61-ac6a-aca521cdaec2 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:34.531438 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e89c26-9c04-4c61-ac6a-aca521cdaec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:34.551418 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:34.608704 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-d0e89c26-9c04-4c61-ac6a-aca521cdaec2 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:34.609903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d0e89c26-9c04-4c61-ac6a-aca521cdaec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:34.703345 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 549/1101] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 21:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:14:34.854886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90efbc8-6889-4e1d-8bf8-ec506d25b258 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:34.856404 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d90efbc8-6889-4e1d-8bf8-ec506d25b258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:34.966972 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:35.003453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90efbc8-6889-4e1d-8bf8-ec506d25b258 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:35.116782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee9b8c-fcb0-4a7e-b11c-9a7b155cbc4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:35.117299 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee9b8c-fcb0-4a7e-b11c-9a7b155cbc4a service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:35.117571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee9b8c-fcb0-4a7e-b11c-9a7b155cbc4a service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:35.117866 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee9b8c-fcb0-4a7e-b11c-9a7b155cbc4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:35.119536 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c2ee9b8c-fcb0-4a7e-b11c-9a7b155cbc4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:35.197969 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:35.212176 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee9b8c-fcb0-4a7e-b11c-9a7b155cbc4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:35.213796 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee9b8c-fcb0-4a7e-b11c-9a7b155cbc4a service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:35.234281 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 553/1102] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:36.430825 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97724f-e14c-4a58-aaa6-00ca4726b686 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:36.431395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97724f-e14c-4a58-aaa6-00ca4726b686 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:36.431395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97724f-e14c-4a58-aaa6-00ca4726b686 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:36.432747 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 550/1103] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 21:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:36.438958 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787d653-a8cb-41ab-bf77-1128826a5e11 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:36.439445 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787d653-a8cb-41ab-bf77-1128826a5e11 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:36.439549 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787d653-a8cb-41ab-bf77-1128826a5e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:36.519750 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-b787d653-a8cb-41ab-bf77-1128826a5e11 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:36.520950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-b787d653-a8cb-41ab-bf77-1128826a5e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:36.618105 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 554/1104] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 21:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:36.770388 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97724f-e14c-4a58-aaa6-00ca4726b686 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:36.771934 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-4c97724f-e14c-4a58-aaa6-00ca4726b686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:36.857398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:36.917627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97724f-e14c-4a58-aaa6-00ca4726b686 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:37.026762 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba6a6c2-74b5-4766-a860-973f33aa22ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:37.027370 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6a6c2-74b5-4766-a860-973f33aa22ed service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:37.027700 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6a6c2-74b5-4766-a860-973f33aa22ed service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:37.027977 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6a6c2-74b5-4766-a860-973f33aa22ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:37.030050 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-cba6a6c2-74b5-4766-a860-973f33aa22ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:37.079460 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:37.120522 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba6a6c2-74b5-4766-a860-973f33aa22ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:37.122114 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba6a6c2-74b5-4766-a860-973f33aa22ed service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:37.140231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 551/1105] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:38.192284 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa413562-08f6-40ff-bf72-6d16eb366be4 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:38.192809 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa413562-08f6-40ff-bf72-6d16eb366be4 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:38.192809 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa413562-08f6-40ff-bf72-6d16eb366be4 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:38.194058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 555/1106] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 21:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:38.200292 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283f0c4-1c03-4caa-979e-97604197be8f None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:38.200730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283f0c4-1c03-4caa-979e-97604197be8f None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:38.200909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283f0c4-1c03-4caa-979e-97604197be8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:38.279987 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-9283f0c4-1c03-4caa-979e-97604197be8f None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:38.281225 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9283f0c4-1c03-4caa-979e-97604197be8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:38.375773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 552/1107] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 21:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:38.528017 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa413562-08f6-40ff-bf72-6d16eb366be4 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:38.529528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-aa413562-08f6-40ff-bf72-6d16eb366be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:38.578386 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:38.673799 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa413562-08f6-40ff-bf72-6d16eb366be4 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:38.785376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc57d1e-b992-46ce-bbc9-37bcb08f5b5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:38.785903 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc57d1e-b992-46ce-bbc9-37bcb08f5b5e service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:38.786179 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc57d1e-b992-46ce-bbc9-37bcb08f5b5e service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:38.786445 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc57d1e-b992-46ce-bbc9-37bcb08f5b5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:38.788174 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-0dc57d1e-b992-46ce-bbc9-37bcb08f5b5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:38.796486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:38.877833 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc57d1e-b992-46ce-bbc9-37bcb08f5b5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:38.879716 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc57d1e-b992-46ce-bbc9-37bcb08f5b5e service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:38.897670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 556/1108] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:39.955934 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e7fb0-d3f8-4232-a58e-64e6de95513d None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:39.956512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e7fb0-d3f8-4232-a58e-64e6de95513d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:39.956512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e7fb0-d3f8-4232-a58e-64e6de95513d None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:39.957840 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 553/1109] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 21:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:39.964250 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bd803-08c9-41aa-93e0-cd7807a7d03e None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:39.964676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bd803-08c9-41aa-93e0-cd7807a7d03e None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:39.964788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bd803-08c9-41aa-93e0-cd7807a7d03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:40.041279 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-d71bd803-08c9-41aa-93e0-cd7807a7d03e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:40.042385 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d71bd803-08c9-41aa-93e0-cd7807a7d03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:40.109870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:40.138735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 557/1110] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 21:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:40.289688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e7fb0-d3f8-4232-a58e-64e6de95513d None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:40.291201 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-272e7fb0-d3f8-4232-a58e-64e6de95513d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:40.308861 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:40.434061 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e7fb0-d3f8-4232-a58e-64e6de95513d None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:40.524248 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:40.549430 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81f30f-1247-4f93-9f7c-c6e433ef5c68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:40.550039 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81f30f-1247-4f93-9f7c-c6e433ef5c68 service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:40.550317 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81f30f-1247-4f93-9f7c-c6e433ef5c68 service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:40.550611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81f30f-1247-4f93-9f7c-c6e433ef5c68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:40.552264 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-7a81f30f-1247-4f93-9f7c-c6e433ef5c68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:40.640862 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a81f30f-1247-4f93-9f7c-c6e433ef5c68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:40.642375 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a81f30f-1247-4f93-9f7c-c6e433ef5c68 service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:40.660467 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 554/1111] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:41.784135 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89833683-753f-4105-9072-6f0d62dd1f37 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:41.784678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89833683-753f-4105-9072-6f0d62dd1f37 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:41.784678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89833683-753f-4105-9072-6f0d62dd1f37 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:41.785996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 558/1112] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 21:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:41.792171 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ca86b-f64b-4ce8-b0ce-1843c3b643c4 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:41.792591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ca86b-f64b-4ce8-b0ce-1843c3b643c4 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:41.792708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ca86b-f64b-4ce8-b0ce-1843c3b643c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:41.866165 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-d48ca86b-f64b-4ce8-b0ce-1843c3b643c4 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:41.867379 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d48ca86b-f64b-4ce8-b0ce-1843c3b643c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:41.908746 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:41.961900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 555/1113] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 21:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:42.110740 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89833683-753f-4105-9072-6f0d62dd1f37 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:42.112243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-89833683-753f-4105-9072-6f0d62dd1f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:42.258352 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89833683-753f-4105-9072-6f0d62dd1f37 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:42.312004 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:42.370123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35e769-af80-47f7-9437-aaab1e2deb7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:42.370658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35e769-af80-47f7-9437-aaab1e2deb7e service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:42.370930 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35e769-af80-47f7-9437-aaab1e2deb7e service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:42.371195 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35e769-af80-47f7-9437-aaab1e2deb7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:42.373369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3c35e769-af80-47f7-9437-aaab1e2deb7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:42.460184 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35e769-af80-47f7-9437-aaab1e2deb7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:42.461746 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35e769-af80-47f7-9437-aaab1e2deb7e service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:42.480489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 559/1114] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:43.526507 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbfebe-20f1-41aa-b1e2-0ff77dd7573d None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:43.527104 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbfebe-20f1-41aa-b1e2-0ff77dd7573d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:43.527104 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbfebe-20f1-41aa-b1e2-0ff77dd7573d None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:43.528985 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 556/1115] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 21:14:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:43.535138 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3552cf-2812-4df4-a8a5-cf22c691801e None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:43.535617 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3552cf-2812-4df4-a8a5-cf22c691801e None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:43.535714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3552cf-2812-4df4-a8a5-cf22c691801e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:43.612670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-ef3552cf-2812-4df4-a8a5-cf22c691801e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:43.613902 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ef3552cf-2812-4df4-a8a5-cf22c691801e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:43.615941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:43.711437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 560/1116] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 21:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:43.863463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbfebe-20f1-41aa-b1e2-0ff77dd7573d None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:43.864913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-45cbfebe-20f1-41aa-b1e2-0ff77dd7573d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:44.009131 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbfebe-20f1-41aa-b1e2-0ff77dd7573d None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:44.035950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:44.120190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70608faf-f36d-464a-b55a-aa54050817bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:44.120713 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70608faf-f36d-464a-b55a-aa54050817bc service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:44.120988 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70608faf-f36d-464a-b55a-aa54050817bc service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:44.121258 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70608faf-f36d-464a-b55a-aa54050817bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:44.123426 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-70608faf-f36d-464a-b55a-aa54050817bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:44.213473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70608faf-f36d-464a-b55a-aa54050817bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:44.215063 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70608faf-f36d-464a-b55a-aa54050817bc service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:44.233618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 557/1117] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:46.278393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcee05-5947-4d4e-8532-e6ed6d483e3d None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:46.278963 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcee05-5947-4d4e-8532-e6ed6d483e3d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:46.278963 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcee05-5947-4d4e-8532-e6ed6d483e3d None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:46.280394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 561/1118] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 21:14:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:46.285802 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320af6b-e690-49dd-8bcd-1ea6abde7777 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:46.286245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320af6b-e690-49dd-8bcd-1ea6abde7777 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:46.286343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320af6b-e690-49dd-8bcd-1ea6abde7777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:46.327462 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:46.356131 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-8320af6b-e690-49dd-8bcd-1ea6abde7777 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:46.357349 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8320af6b-e690-49dd-8bcd-1ea6abde7777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:46.451248 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 558/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 21:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:46.733695 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bcee05-5947-4d4e-8532-e6ed6d483e3d None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:46.735295 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-69bcee05-5947-4d4e-8532-e6ed6d483e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:46.836887 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bcee05-5947-4d4e-8532-e6ed6d483e3d None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:46.868288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:46.913387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24aaed-e0c3-4fae-aeb1-f43b26bb2997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:46.913984 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24aaed-e0c3-4fae-aeb1-f43b26bb2997 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:46.914256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24aaed-e0c3-4fae-aeb1-f43b26bb2997 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:46.914517 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24aaed-e0c3-4fae-aeb1-f43b26bb2997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:46.916107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ad24aaed-e0c3-4fae-aeb1-f43b26bb2997 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:47.021857 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24aaed-e0c3-4fae-aeb1-f43b26bb2997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:47.023273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24aaed-e0c3-4fae-aeb1-f43b26bb2997 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:47.040634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 562/1120] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:14:49.353649 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f6d58-b8c5-4a30-9d8e-f804b39e3850 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:49.354251 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f6d58-b8c5-4a30-9d8e-f804b39e3850 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:49.354251 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f6d58-b8c5-4a30-9d8e-f804b39e3850 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:49.355515 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 559/1121] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 21:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:49.360621 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff981f7-f325-4499-b2c3-06027b15842b None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:49.361067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff981f7-f325-4499-b2c3-06027b15842b None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:49.361173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff981f7-f325-4499-b2c3-06027b15842b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:49.427452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-3ff981f7-f325-4499-b2c3-06027b15842b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:49.428656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3ff981f7-f325-4499-b2c3-06027b15842b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:49.441061 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:49.526614 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 563/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:49.810489 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f6d58-b8c5-4a30-9d8e-f804b39e3850 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:49.812333 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-260f6d58-b8c5-4a30-9d8e-f804b39e3850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:49.914065 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f6d58-b8c5-4a30-9d8e-f804b39e3850 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:49.926770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:49.986412 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437aaf54-9c2d-43fd-a06d-6e0e1d11e1d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:49.987065 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437aaf54-9c2d-43fd-a06d-6e0e1d11e1d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:49.987299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437aaf54-9c2d-43fd-a06d-6e0e1d11e1d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:49.987578 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437aaf54-9c2d-43fd-a06d-6e0e1d11e1d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:49.989159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-437aaf54-9c2d-43fd-a06d-6e0e1d11e1d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:50.094249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437aaf54-9c2d-43fd-a06d-6e0e1d11e1d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:50.095708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437aaf54-9c2d-43fd-a06d-6e0e1d11e1d9 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:50.115504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 560/1123] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:52.557247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7712a31-9310-4480-9545-1147a75e9b89 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:52.557809 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7712a31-9310-4480-9545-1147a75e9b89 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:52.557809 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7712a31-9310-4480-9545-1147a75e9b89 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:52.559159 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 564/1124] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 21:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:52.564383 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dab529-a74e-4241-920c-4300728db8e0 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:52.564777 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dab529-a74e-4241-920c-4300728db8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:52.564925 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dab529-a74e-4241-920c-4300728db8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:52.611642 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:52.629707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-a8dab529-a74e-4241-920c-4300728db8e0 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:52.630896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-a8dab529-a74e-4241-920c-4300728db8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:52.723556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 561/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 21:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:53.012748 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7712a31-9310-4480-9545-1147a75e9b89 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:53.014286 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-e7712a31-9310-4480-9545-1147a75e9b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:53.116081 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7712a31-9310-4480-9545-1147a75e9b89 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:53.155873 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:53.189565 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1fd72-4a3f-4e91-81c0-4b741f76cbe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:53.190219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1fd72-4a3f-4e91-81c0-4b741f76cbe1 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:53.190499 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1fd72-4a3f-4e91-81c0-4b741f76cbe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:53.190766 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1fd72-4a3f-4e91-81c0-4b741f76cbe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:53.192367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-64f1fd72-4a3f-4e91-81c0-4b741f76cbe1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:53.296838 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1fd72-4a3f-4e91-81c0-4b741f76cbe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:53.298142 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1fd72-4a3f-4e91-81c0-4b741f76cbe1 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:53.315822 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 565/1126] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:55.585024 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293bb74-086b-409d-b82c-b3243ef7854e None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:55.585590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293bb74-086b-409d-b82c-b3243ef7854e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:55.585590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293bb74-086b-409d-b82c-b3243ef7854e None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:55.586890 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 562/1127] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 21:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:55.592611 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f0712-54a4-430c-8db6-3ebda8c2f27c None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:55.592998 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f0712-54a4-430c-8db6-3ebda8c2f27c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:55.593161 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f0712-54a4-430c-8db6-3ebda8c2f27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:55.657941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-226f0712-54a4-430c-8db6-3ebda8c2f27c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:55.659134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-226f0712-54a4-430c-8db6-3ebda8c2f27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:55.679442 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:55.758904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 566/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 21:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:56.041027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293bb74-086b-409d-b82c-b3243ef7854e None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:56.043089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-c293bb74-086b-409d-b82c-b3243ef7854e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:56.143473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293bb74-086b-409d-b82c-b3243ef7854e None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:56.163848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:56.217048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02232599-4723-4eef-a954-bc613ea5768d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:56.217652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02232599-4723-4eef-a954-bc613ea5768d service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:56.217925 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02232599-4723-4eef-a954-bc613ea5768d service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:56.218193 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02232599-4723-4eef-a954-bc613ea5768d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:56.219783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-02232599-4723-4eef-a954-bc613ea5768d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:56.329616 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02232599-4723-4eef-a954-bc613ea5768d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:56.330956 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02232599-4723-4eef-a954-bc613ea5768d service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:56.350978 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 563/1129] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:14:58.546007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7550675-0a04-4e3b-9319-52a92085fda0 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:58.546536 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7550675-0a04-4e3b-9319-52a92085fda0 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:58.546536 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7550675-0a04-4e3b-9319-52a92085fda0 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:58.547929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 567/1130] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 21:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:14:58.553184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28e1d4-1351-4798-b179-be0d654009c2 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:58.553624 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28e1d4-1351-4798-b179-be0d654009c2 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:58.553746 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28e1d4-1351-4798-b179-be0d654009c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:58.617187 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:58.619868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-8a28e1d4-1351-4798-b179-be0d654009c2 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:14:58.622249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8a28e1d4-1351-4798-b179-be0d654009c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:58.715796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 564/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 21:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:14:59.000036 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7550675-0a04-4e3b-9319-52a92085fda0 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:14:59.001602 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a7550675-0a04-4e3b-9319-52a92085fda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:59.101823 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7550675-0a04-4e3b-9319-52a92085fda0 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:14:59.150316 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:14:59.174078 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde451a8-4d3c-4012-b6e9-4245694a1de2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:14:59.174678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde451a8-4d3c-4012-b6e9-4245694a1de2 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:14:59.174954 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde451a8-4d3c-4012-b6e9-4245694a1de2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:14:59.175222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde451a8-4d3c-4012-b6e9-4245694a1de2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:14:59.176823 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-cde451a8-4d3c-4012-b6e9-4245694a1de2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:14:59.282875 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde451a8-4d3c-4012-b6e9-4245694a1de2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:14:59.284246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde451a8-4d3c-4012-b6e9-4245694a1de2 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:14:59.302032 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 568/1132] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:14:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:15:01.493256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e625722-c8be-446f-bfd7-5aa3cceb8a96 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:01.493808 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e625722-c8be-446f-bfd7-5aa3cceb8a96 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:01.493808 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e625722-c8be-446f-bfd7-5aa3cceb8a96 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:01.495137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 565/1133] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 21:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:15:01.501056 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064954ba-b157-4119-9648-7b3b5a710c87 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:01.501516 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064954ba-b157-4119-9648-7b3b5a710c87 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:01.501622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064954ba-b157-4119-9648-7b3b5a710c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:01.566536 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-064954ba-b157-4119-9648-7b3b5a710c87 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:15:01.567760 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-064954ba-b157-4119-9648-7b3b5a710c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:01.596362 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:01.664168 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 569/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 21:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:15:01.950200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e625722-c8be-446f-bfd7-5aa3cceb8a96 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:15:01.951785 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-7e625722-c8be-446f-bfd7-5aa3cceb8a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:02.051252 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e625722-c8be-446f-bfd7-5aa3cceb8a96 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:15:02.078878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:02.122506 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98650c-dd22-420a-948a-f1d24faecff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:15:02.123104 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98650c-dd22-420a-948a-f1d24faecff6 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:02.123382 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98650c-dd22-420a-948a-f1d24faecff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:02.123682 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98650c-dd22-420a-948a-f1d24faecff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:02.125259 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9a98650c-dd22-420a-948a-f1d24faecff6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:02.229063 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98650c-dd22-420a-948a-f1d24faecff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:15:02.230512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98650c-dd22-420a-948a-f1d24faecff6 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:15:02.248228 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 566/1135] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:15:04.432745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bfb4d-3916-4771-abc5-92a1b5609eea None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:04.433296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bfb4d-3916-4771-abc5-92a1b5609eea None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:04.433296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bfb4d-3916-4771-abc5-92a1b5609eea None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:04.434597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 570/1136] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 21:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:15:04.439951 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3f411-3e6d-4d69-8a6b-e16334cbb019 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:04.440374 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3f411-3e6d-4d69-8a6b-e16334cbb019 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:04.440506 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3f411-3e6d-4d69-8a6b-e16334cbb019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:04.509677 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-53b3f411-3e6d-4d69-8a6b-e16334cbb019 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:15:04.511544 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-53b3f411-3e6d-4d69-8a6b-e16334cbb019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:04.519723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:04.606518 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 567/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 21:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:15:04.888259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bfb4d-3916-4771-abc5-92a1b5609eea None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:15:04.889813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-095bfb4d-3916-4771-abc5-92a1b5609eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:04.992553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bfb4d-3916-4771-abc5-92a1b5609eea None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:15:05.048046 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:05.067539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a09cfe-99ac-4ca4-88f7-06676b6a6fa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:15:05.068175 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a09cfe-99ac-4ca4-88f7-06676b6a6fa3 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:05.068450 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a09cfe-99ac-4ca4-88f7-06676b6a6fa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:05.068721 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a09cfe-99ac-4ca4-88f7-06676b6a6fa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:05.070322 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-86a09cfe-99ac-4ca4-88f7-06676b6a6fa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:05.177678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a09cfe-99ac-4ca4-88f7-06676b6a6fa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:15:05.179102 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a09cfe-99ac-4ca4-88f7-06676b6a6fa3 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:15:05.196627 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 571/1138] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:15:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:15:07.418293 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16879af-c3aa-44f2-b4c1-832b2065c4d4 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:07.418995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16879af-c3aa-44f2-b4c1-832b2065c4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:07.418995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16879af-c3aa-44f2-b4c1-832b2065c4d4 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:07.420254 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 568/1139] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 21:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:15:07.425647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82f4f6-7523-4ed5-9c89-b0019dab584c None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:07.426103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82f4f6-7523-4ed5-9c89-b0019dab584c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:07.426202 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82f4f6-7523-4ed5-9c89-b0019dab584c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:07.494341 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-eb82f4f6-7523-4ed5-9c89-b0019dab584c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:15:07.496179 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-eb82f4f6-7523-4ed5-9c89-b0019dab584c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:07.534842 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:07.597103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 572/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:15:07.882648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16879af-c3aa-44f2-b4c1-832b2065c4d4 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:15:07.884208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b16879af-c3aa-44f2-b4c1-832b2065c4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:07.987514 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16879af-c3aa-44f2-b4c1-832b2065c4d4 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:15:08.024180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:08.058709 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff07129-cedd-4f3b-8faa-251c5eab1532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:15:08.059302 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff07129-cedd-4f3b-8faa-251c5eab1532 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:08.059596 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff07129-cedd-4f3b-8faa-251c5eab1532 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:08.059865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff07129-cedd-4f3b-8faa-251c5eab1532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:08.061427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-3ff07129-cedd-4f3b-8faa-251c5eab1532 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:08.167244 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff07129-cedd-4f3b-8faa-251c5eab1532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:15:08.168713 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff07129-cedd-4f3b-8faa-251c5eab1532 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:15:08.186128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 569/1141] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:15:10.347947 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3606f6-009e-465b-8cdd-7b1fa5fbe198 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:10.348546 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3606f6-009e-465b-8cdd-7b1fa5fbe198 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:10.348546 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3606f6-009e-465b-8cdd-7b1fa5fbe198 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:10.349835 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 573/1142] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 21:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:15:10.355396 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685838ee-2c23-4e72-9328-6c8484c35132 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:10.355843 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685838ee-2c23-4e72-9328-6c8484c35132 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:10.355843 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685838ee-2c23-4e72-9328-6c8484c35132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:10.421031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-685838ee-2c23-4e72-9328-6c8484c35132 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:15:10.422227 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-685838ee-2c23-4e72-9328-6c8484c35132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:10.439657 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:10.520994 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 570/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 21:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:15:10.805849 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3606f6-009e-465b-8cdd-7b1fa5fbe198 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:15:10.807449 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ab3606f6-009e-465b-8cdd-7b1fa5fbe198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:10.911870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3606f6-009e-465b-8cdd-7b1fa5fbe198 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:15:10.976719 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:10.983413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe276b-b0a6-40f8-a5fd-2facaeb9c71b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:15:10.984065 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe276b-b0a6-40f8-a5fd-2facaeb9c71b service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:10.984339 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe276b-b0a6-40f8-a5fd-2facaeb9c71b service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:10.984604 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe276b-b0a6-40f8-a5fd-2facaeb9c71b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:10.986204 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ddfe276b-b0a6-40f8-a5fd-2facaeb9c71b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:11.091101 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe276b-b0a6-40f8-a5fd-2facaeb9c71b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:15:11.092442 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe276b-b0a6-40f8-a5fd-2facaeb9c71b service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:15:11.110000 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 574/1144] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:15:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:15:13.258245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb26844-aa9b-454c-bf99-9fce1ceb77a0 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:13.258831 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb26844-aa9b-454c-bf99-9fce1ceb77a0 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:13.258831 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb26844-aa9b-454c-bf99-9fce1ceb77a0 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:13.260155 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 571/1145] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 21:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:15:13.265465 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65956ea-daf2-4ea2-b131-b624c60f55c3 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:13.265853 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65956ea-daf2-4ea2-b131-b624c60f55c3 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:13.266009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65956ea-daf2-4ea2-b131-b624c60f55c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:13.330737 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-e65956ea-daf2-4ea2-b131-b624c60f55c3 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:15:13.331976 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-e65956ea-daf2-4ea2-b131-b624c60f55c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:13.380360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:13.428726 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 575/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 21:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:15:13.712704 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb26844-aa9b-454c-bf99-9fce1ceb77a0 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:15:13.714765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-cdb26844-aa9b-454c-bf99-9fce1ceb77a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:13.818912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb26844-aa9b-454c-bf99-9fce1ceb77a0 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:15:13.867182 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:13.893399 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df32dc-8af5-4d48-b209-019edfd9039d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:15:13.893991 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df32dc-8af5-4d48-b209-019edfd9039d service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:13.894271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df32dc-8af5-4d48-b209-019edfd9039d service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:13.894559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df32dc-8af5-4d48-b209-019edfd9039d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:13.896145 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-74df32dc-8af5-4d48-b209-019edfd9039d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:14.003636 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df32dc-8af5-4d48-b209-019edfd9039d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:15:14.004976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df32dc-8af5-4d48-b209-019edfd9039d service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:15:14.022986 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 572/1147] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:15:23.427196 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=22 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:24.020856 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=66 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:33.936474 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed334977-92d5-4f35-9b15-b85a04ccadee None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:33.937000 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed334977-92d5-4f35-9b15-b85a04ccadee None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:33.937000 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed334977-92d5-4f35-9b15-b85a04ccadee None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:33.938269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 576/1148] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:15:33.943390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90583e0-27a5-467c-828c-ce6b825919e9 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:33.943870 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90583e0-27a5-467c-828c-ce6b825919e9 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:33.943983 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90583e0-27a5-467c-828c-ce6b825919e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:34.007917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-d90583e0-27a5-467c-828c-ce6b825919e9 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:15:34.009112 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d90583e0-27a5-467c-828c-ce6b825919e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:34.104222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 573/1149] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:15:34.401954 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca120d97-9abb-460d-b80e-802d317c6e94 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:34.402235 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca120d97-9abb-460d-b80e-802d317c6e94 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:34.402522 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca120d97-9abb-460d-b80e-802d317c6e94 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:34.403926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 577/1150] 10.222.0.13 () {54 vars in 962 bytes} [Fri Oct 21 21:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:15:34.409042 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6e5e7-7511-4e03-914a-c059899631ef None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:34.409449 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6e5e7-7511-4e03-914a-c059899631ef None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:34.409733 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6e5e7-7511-4e03-914a-c059899631ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:34.484881 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-3dd6e5e7-7511-4e03-914a-c059899631ef None None] MFA Rules not processed for user `5ce5ec441b06408bbadd6fc040af0da5`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:15:34.486059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-3dd6e5e7-7511-4e03-914a-c059899631ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:34.506322 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:34.560199 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 574/1151] 10.222.0.13 () {58 vars in 1059 bytes} [Fri Oct 21 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:15:34.567852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca120d97-9abb-460d-b80e-802d317c6e94 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:15:34.569386 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ca120d97-9abb-460d-b80e-802d317c6e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:34.674067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca120d97-9abb-460d-b80e-802d317c6e94 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:15:34.747109 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5bc7a7-ea35-46b4-ab51-8890ee23a345 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce5ec441b06408bbadd6fc040af0da5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:15:34.747744 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bc7a7-ea35-46b4-ab51-8890ee23a345 service glance] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:34.748017 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bc7a7-ea35-46b4-ab51-8890ee23a345 service glance] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:34.748277 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bc7a7-ea35-46b4-ab51-8890ee23a345 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:34.749833 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ca5bc7a7-ea35-46b4-ab51-8890ee23a345 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:34.802497 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:34.857290 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5bc7a7-ea35-46b4-ab51-8890ee23a345 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:15:34.858810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5bc7a7-ea35-46b4-ab51-8890ee23a345 service glance] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:15:34.877308 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 578/1152] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:15:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:15:37.116573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760376a-1cd0-4aad-9c87-dd7f93e2c166 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:37.117139 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760376a-1cd0-4aad-9c87-dd7f93e2c166 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:37.117139 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760376a-1cd0-4aad-9c87-dd7f93e2c166 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:37.118368 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 575/1153] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:15:37.124281 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979666f-95fe-4140-94df-cd8de17ba81b None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:37.124688 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979666f-95fe-4140-94df-cd8de17ba81b None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:37.124828 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979666f-95fe-4140-94df-cd8de17ba81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:37.189095 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-5979666f-95fe-4140-94df-cd8de17ba81b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:15:37.190293 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-5979666f-95fe-4140-94df-cd8de17ba81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:37.282867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 579/1154] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:15:37.562416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760376a-1cd0-4aad-9c87-dd7f93e2c166 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:15:37.564015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f760376a-1cd0-4aad-9c87-dd7f93e2c166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:37.686720 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760376a-1cd0-4aad-9c87-dd7f93e2c166 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:15:37.735184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:37.780469 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f421fa4-f575-4b7f-9809-703d309daaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:15:37.781071 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f421fa4-f575-4b7f-9809-703d309daaaa admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:37.781348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f421fa4-f575-4b7f-9809-703d309daaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:37.781619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f421fa4-f575-4b7f-9809-703d309daaaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:37.782242 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f421fa4-f575-4b7f-9809-703d309daaaa admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:15:37.783341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: WARNING py.warnings [None req-6f421fa4-f575-4b7f-9809-703d309daaaa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 21:15:37.783341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: warnings.warn(msg) Oct 21 21:15:37.783341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: Oct 21 21:15:37.784509 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f421fa4-f575-4b7f-9809-703d309daaaa admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:15:37.802979 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 576/1155] 10.222.0.13 () {56 vars in 1108 bytes} [Fri Oct 21 21:15:37 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:15:39.953477 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02899893-5008-4377-bf9d-62398e9e05a3 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:39.954027 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02899893-5008-4377-bf9d-62398e9e05a3 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:39.954027 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02899893-5008-4377-bf9d-62398e9e05a3 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:39.955427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 580/1156] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:15:39.960835 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e80903-a738-487c-941f-052677f354cf None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:39.961232 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e80903-a738-487c-941f-052677f354cf None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:39.961419 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e80903-a738-487c-941f-052677f354cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:40.027633 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-00e80903-a738-487c-941f-052677f354cf None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:15:40.029271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-00e80903-a738-487c-941f-052677f354cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:40.121493 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 577/1157] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:15:40.416462 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02899893-5008-4377-bf9d-62398e9e05a3 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:15:40.418038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-02899893-5008-4377-bf9d-62398e9e05a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:40.428326 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:40.567803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02899893-5008-4377-bf9d-62398e9e05a3 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:15:40.645977 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:40.678867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195909d6-6601-4033-a965-b15b69ac1c57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:15:40.679473 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195909d6-6601-4033-a965-b15b69ac1c57 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:40.679783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195909d6-6601-4033-a965-b15b69ac1c57 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:40.680059 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195909d6-6601-4033-a965-b15b69ac1c57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:40.681653 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-195909d6-6601-4033-a965-b15b69ac1c57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:40.774584 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195909d6-6601-4033-a965-b15b69ac1c57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:15:40.776223 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195909d6-6601-4033-a965-b15b69ac1c57 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:15:40.794920 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 581/1158] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:15:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:15:43.118792 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7a8bf-59c6-4f8d-ae1f-886f75e16f7d None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:43.119322 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7a8bf-59c6-4f8d-ae1f-886f75e16f7d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:43.119322 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7a8bf-59c6-4f8d-ae1f-886f75e16f7d None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:43.120643 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 578/1159] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:15:43.126338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc03ffc-0ea3-4988-8848-a24f8df213ac None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:43.126767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc03ffc-0ea3-4988-8848-a24f8df213ac None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:43.126870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc03ffc-0ea3-4988-8848-a24f8df213ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:43.191452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-bbc03ffc-0ea3-4988-8848-a24f8df213ac None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:15:43.192667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-bbc03ffc-0ea3-4988-8848-a24f8df213ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:43.237666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:43.287578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 582/1160] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:15:43.572659 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7a8bf-59c6-4f8d-ae1f-886f75e16f7d None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:15:43.574260 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-98f7a8bf-59c6-4f8d-ae1f-886f75e16f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:43.597678 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:43.674607 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7a8bf-59c6-4f8d-ae1f-886f75e16f7d None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:15:43.747885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dede5b-bd8b-4c21-ad30-2acb394e4c0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:15:43.748495 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dede5b-bd8b-4c21-ad30-2acb394e4c0c service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:43.748773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dede5b-bd8b-4c21-ad30-2acb394e4c0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:43.749041 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dede5b-bd8b-4c21-ad30-2acb394e4c0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:43.750611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-66dede5b-bd8b-4c21-ad30-2acb394e4c0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:43.823427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:43.858251 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dede5b-bd8b-4c21-ad30-2acb394e4c0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:15:43.859710 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dede5b-bd8b-4c21-ad30-2acb394e4c0c service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:15:43.876896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 579/1161] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:15:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:15:46.067893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f845e2e-0b49-4b22-8230-031f5f599cc4 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:46.068406 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f845e2e-0b49-4b22-8230-031f5f599cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:46.068406 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f845e2e-0b49-4b22-8230-031f5f599cc4 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:46.069700 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 583/1162] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:15:46.075201 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d78d2b-0cb5-4350-8485-87228cdd892c None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:46.076215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d78d2b-0cb5-4350-8485-87228cdd892c None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:46.076348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d78d2b-0cb5-4350-8485-87228cdd892c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:46.141180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-14d78d2b-0cb5-4350-8485-87228cdd892c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:15:46.142380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-14d78d2b-0cb5-4350-8485-87228cdd892c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:46.235662 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 580/1163] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:15:46.525512 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f845e2e-0b49-4b22-8230-031f5f599cc4 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:15:46.527047 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1f845e2e-0b49-4b22-8230-031f5f599cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:46.628883 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f845e2e-0b49-4b22-8230-031f5f599cc4 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:15:46.701655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:46.703128 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31188e45-644d-4598-82f8-ec75e411cb22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:15:46.703775 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31188e45-644d-4598-82f8-ec75e411cb22 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:46.704266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31188e45-644d-4598-82f8-ec75e411cb22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:46.704745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31188e45-644d-4598-82f8-ec75e411cb22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:46.706560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-31188e45-644d-4598-82f8-ec75e411cb22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:46.817583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31188e45-644d-4598-82f8-ec75e411cb22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:15:46.819007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31188e45-644d-4598-82f8-ec75e411cb22 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:15:46.836888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 584/1164] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:15:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:15:49.140948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae96e1-3a3f-476a-8cd5-5d8aecf05354 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:49.141473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae96e1-3a3f-476a-8cd5-5d8aecf05354 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:49.141473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae96e1-3a3f-476a-8cd5-5d8aecf05354 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:49.142746 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 581/1165] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:15:49.148043 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e32af-3ef8-4ee0-98f5-1428ae6beb57 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:49.148451 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e32af-3ef8-4ee0-98f5-1428ae6beb57 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:49.148577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e32af-3ef8-4ee0-98f5-1428ae6beb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:49.212448 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-596e32af-3ef8-4ee0-98f5-1428ae6beb57 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:15:49.213629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-596e32af-3ef8-4ee0-98f5-1428ae6beb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:49.265052 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:49.308259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 585/1166] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:15:49.596536 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae96e1-3a3f-476a-8cd5-5d8aecf05354 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:15:49.598072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-12ae96e1-3a3f-476a-8cd5-5d8aecf05354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:49.634002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:49.699920 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae96e1-3a3f-476a-8cd5-5d8aecf05354 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:15:49.773734 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafad56-ec9c-427e-8f2c-79cfea500692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:15:49.774341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafad56-ec9c-427e-8f2c-79cfea500692 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:49.774617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafad56-ec9c-427e-8f2c-79cfea500692 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:49.774888 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafad56-ec9c-427e-8f2c-79cfea500692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:49.776520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-7eafad56-ec9c-427e-8f2c-79cfea500692 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:49.855215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:49.883949 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafad56-ec9c-427e-8f2c-79cfea500692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:15:49.885434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafad56-ec9c-427e-8f2c-79cfea500692 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:15:49.903435 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 582/1167] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:15:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:15:52.236049 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586babbc-d1e9-4cd8-adf5-fd307738d90b None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:52.236582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586babbc-d1e9-4cd8-adf5-fd307738d90b None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:52.236707 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586babbc-d1e9-4cd8-adf5-fd307738d90b None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:52.238020 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 586/1168] 10.222.0.13 () {54 vars in 927 bytes} [Fri Oct 21 21:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:15:52.243311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb153a-bb95-4b1e-9c91-4f887e2e0ceb None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:52.243798 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb153a-bb95-4b1e-9c91-4f887e2e0ceb None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:52.243895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb153a-bb95-4b1e-9c91-4f887e2e0ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:52.307977 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-d7eb153a-bb95-4b1e-9c91-4f887e2e0ceb None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:15:52.309169 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d7eb153a-bb95-4b1e-9c91-4f887e2e0ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:52.403450 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 583/1169] 10.222.0.13 () {58 vars in 1024 bytes} [Fri Oct 21 21:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:15:52.691644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586babbc-d1e9-4cd8-adf5-fd307738d90b None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:15:52.693183 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-586babbc-d1e9-4cd8-adf5-fd307738d90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:52.798992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586babbc-d1e9-4cd8-adf5-fd307738d90b None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:15:52.872210 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ec1f7-1ba7-416f-b244-bc0756fb0ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:15:52.872835 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ec1f7-1ba7-416f-b244-bc0756fb0ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:15:52.873645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ec1f7-1ba7-416f-b244-bc0756fb0ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:15:52.873913 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ec1f7-1ba7-416f-b244-bc0756fb0ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:15:52.875482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2d8ec1f7-1ba7-416f-b244-bc0756fb0ab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:15:52.884077 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:15:52.981033 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ec1f7-1ba7-416f-b244-bc0756fb0ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:15:52.982928 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ec1f7-1ba7-416f-b244-bc0756fb0ab2 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:15:53.001112 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 587/1170] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:15:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:16:02.401980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=82 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:16:02.999389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=50 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:17:18.763616 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a95cf-4d9d-4c0c-8d04-60aeb16e796d None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:17:18.764155 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a95cf-4d9d-4c0c-8d04-60aeb16e796d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:17:18.764155 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a95cf-4d9d-4c0c-8d04-60aeb16e796d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:17:18.838746 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-264a95cf-4d9d-4c0c-8d04-60aeb16e796d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:17:18.839997 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-264a95cf-4d9d-4c0c-8d04-60aeb16e796d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:17:18.933597 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 584/1171] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:17:18.942136 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e9c5d-1ab7-476e-bce7-605f681ef3bd None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:17:18.942666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e9c5d-1ab7-476e-bce7-605f681ef3bd None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:17:18.942666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e9c5d-1ab7-476e-bce7-605f681ef3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:17:19.017340 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-4d5e9c5d-1ab7-476e-bce7-605f681ef3bd None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:17:19.019029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-4d5e9c5d-1ab7-476e-bce7-605f681ef3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:17:19.113525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 588/1172] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:17:19.118194 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a95cf-4d9d-4c0c-8d04-60aeb16e796d None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:17:19.119408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-264a95cf-4d9d-4c0c-8d04-60aeb16e796d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:17:19.132495 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a95cf-4d9d-4c0c-8d04-60aeb16e796d None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:17:19.192772 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:17:19.225865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a8cc5-90b1-43ed-bd19-8d409c42b0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:17:19.226482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a8cc5-90b1-43ed-bd19-8d409c42b0ee admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:17:19.226763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a8cc5-90b1-43ed-bd19-8d409c42b0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:17:19.227026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a8cc5-90b1-43ed-bd19-8d409c42b0ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:17:19.227684 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a8cc5-90b1-43ed-bd19-8d409c42b0ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:17:19.227987 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a8cc5-90b1-43ed-bd19-8d409c42b0ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:17:19.229012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: WARNING py.warnings [None req-ab1a8cc5-90b1-43ed-bd19-8d409c42b0ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 21 21:17:19.229012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: warnings.warn(msg) Oct 21 21:17:19.229012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: Oct 21 21:17:19.229893 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a8cc5-90b1-43ed-bd19-8d409c42b0ee admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:17:19.242760 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 585/1173] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:17:19.252812 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e9c5d-1ab7-476e-bce7-605f681ef3bd None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:17:19.261801 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e9c5d-1ab7-476e-bce7-605f681ef3bd None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:17:19.322102 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:17:19.354923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae8df1-3b3c-451a-9267-93e1ffaf8d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:17:19.355523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae8df1-3b3c-451a-9267-93e1ffaf8d20 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:17:19.355828 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae8df1-3b3c-451a-9267-93e1ffaf8d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:17:19.356097 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae8df1-3b3c-451a-9267-93e1ffaf8d20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:17:19.356720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae8df1-3b3c-451a-9267-93e1ffaf8d20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:17:19.357808 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: WARNING py.warnings [None req-abae8df1-3b3c-451a-9267-93e1ffaf8d20 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 21:17:19.357808 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: warnings.warn(msg) Oct 21 21:17:19.357808 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: Oct 21 21:17:19.358993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae8df1-3b3c-451a-9267-93e1ffaf8d20 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:17:19.415655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 589/1174] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:17:19 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:17:19.420036 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a8cc5-90b1-43ed-bd19-8d409c42b0ee admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:17:19.421540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ab1a8cc5-90b1-43ed-bd19-8d409c42b0ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:17:19.454032 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a8cc5-90b1-43ed-bd19-8d409c42b0ee admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:17:19.545660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061d291-d89b-4190-a0ac-d2b2193e0087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:17:19.546270 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061d291-d89b-4190-a0ac-d2b2193e0087 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:17:19.546549 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061d291-d89b-4190-a0ac-d2b2193e0087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:17:19.546841 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061d291-d89b-4190-a0ac-d2b2193e0087 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:17:19.547475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061d291-d89b-4190-a0ac-d2b2193e0087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:17:19.548557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: WARNING py.warnings [None req-2061d291-d89b-4190-a0ac-d2b2193e0087 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 21:17:19.548557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: warnings.warn(msg) Oct 21 21:17:19.548557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: Oct 21 21:17:19.549894 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061d291-d89b-4190-a0ac-d2b2193e0087 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:17:19.593482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 586/1175] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:17:19 2022] POST /identity/v3/users => generated 290 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:17:19.597559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae8df1-3b3c-451a-9267-93e1ffaf8d20 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:17:19.599103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-abae8df1-3b3c-451a-9267-93e1ffaf8d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:17:19.660905 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae8df1-3b3c-451a-9267-93e1ffaf8d20 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:17:19.742998 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:17:19.754756 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c405c3-7626-4eae-a8dc-65424d659d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:17:19.755359 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c405c3-7626-4eae-a8dc-65424d659d07 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:17:19.755669 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c405c3-7626-4eae-a8dc-65424d659d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:17:19.755942 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c405c3-7626-4eae-a8dc-65424d659d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:17:19.756581 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c405c3-7626-4eae-a8dc-65424d659d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:17:19.758185 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c405c3-7626-4eae-a8dc-65424d659d07 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:17:19.771180 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 590/1176] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:17:19.775407 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061d291-d89b-4190-a0ac-d2b2193e0087 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:17:19.776943 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2061d291-d89b-4190-a0ac-d2b2193e0087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:17:19.788066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061d291-d89b-4190-a0ac-d2b2193e0087 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:17:19.812673 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:17:19.882628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee5702-fc08-4602-9f77-87a4d53e39fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:17:19.883152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee5702-fc08-4602-9f77-87a4d53e39fd admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:17:19.883429 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee5702-fc08-4602-9f77-87a4d53e39fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:17:19.883726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee5702-fc08-4602-9f77-87a4d53e39fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:17:19.884350 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee5702-fc08-4602-9f77-87a4d53e39fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:17:19.885949 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee5702-fc08-4602-9f77-87a4d53e39fd admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:17:19.898678 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 587/1177] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:17:19.903004 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c405c3-7626-4eae-a8dc-65424d659d07 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:17:19.904625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-e9c405c3-7626-4eae-a8dc-65424d659d07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:17:19.938338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c405c3-7626-4eae-a8dc-65424d659d07 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:17:20.031706 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4da11c-9b71-42ab-b359-a489ca416c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:17:20.032376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4da11c-9b71-42ab-b359-a489ca416c2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:17:20.032631 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4da11c-9b71-42ab-b359-a489ca416c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:17:20.032902 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4da11c-9b71-42ab-b359-a489ca416c2d admin admin] PATH_INFO: `/v3/projects/0b01e31f642542c28286db03a205dbc0/users/ef619fd1c4824c12aa58bdc525f6b28c/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:17:20.054883 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4da11c-9b71-42ab-b359-a489ca416c2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b01e31f642542c28286db03a205dbc0, user_id=ef619fd1c4824c12aa58bdc525f6b28c, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:17:20.057552 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4da11c-9b71-42ab-b359-a489ca416c2d admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:17:20.082206 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 591/1178] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:17:19 2022] PUT /identity/v3/projects/0b01e31f642542c28286db03a205dbc0/users/ef619fd1c4824c12aa58bdc525f6b28c/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:17:20.093075 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70978345-231d-4865-b671-59e78896eac6 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:17:20.093361 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70978345-231d-4865-b671-59e78896eac6 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:17:20.093634 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70978345-231d-4865-b671-59e78896eac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:17:20.149559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-70978345-231d-4865-b671-59e78896eac6 None None] MFA Rules not processed for user `ef619fd1c4824c12aa58bdc525f6b28c`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:17:20.150683 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-70978345-231d-4865-b671-59e78896eac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:17:20.226999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 588/1179] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:17:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:17:20.240361 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a569e1-b129-40b2-bfd6-f8f092708112 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:17:20.240647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a569e1-b129-40b2-bfd6-f8f092708112 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:17:20.240941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a569e1-b129-40b2-bfd6-f8f092708112 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:17:20.242294 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 592/1180] 10.222.0.13 () {54 vars in 963 bytes} [Fri Oct 21 21:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:17:20.247046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145a18d-ee32-4983-ab71-2517b107830d None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:17:20.247439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145a18d-ee32-4983-ab71-2517b107830d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:17:20.247754 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145a18d-ee32-4983-ab71-2517b107830d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:17:20.279611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:17:20.324780 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-0145a18d-ee32-4983-ab71-2517b107830d None None] MFA Rules not processed for user `d620cad0ef21453a9cab61f728913cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:17:20.325975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0145a18d-ee32-4983-ab71-2517b107830d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:17:20.398970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 589/1181] 10.222.0.13 () {58 vars in 1060 bytes} [Fri Oct 21 21:17:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:17:20.409270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a569e1-b129-40b2-bfd6-f8f092708112 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:17:20.410717 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d8a569e1-b129-40b2-bfd6-f8f092708112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:17:20.459813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:17:20.514593 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a569e1-b129-40b2-bfd6-f8f092708112 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:17:20.592959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08b396-a9a7-4c8c-9bf9-566e7c67b6ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:17:20.593555 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08b396-a9a7-4c8c-9bf9-566e7c67b6ff service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:17:20.593826 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08b396-a9a7-4c8c-9bf9-566e7c67b6ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:17:20.594090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08b396-a9a7-4c8c-9bf9-566e7c67b6ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:17:20.595694 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ec08b396-a9a7-4c8c-9bf9-566e7c67b6ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:17:20.692215 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08b396-a9a7-4c8c-9bf9-566e7c67b6ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:17:20.693539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08b396-a9a7-4c8c-9bf9-566e7c67b6ff service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:17:20.697055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:17:20.712025 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 593/1182] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:17:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:17:20.758753 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217e4c8-3a7d-4b7d-bfe8-9b5dd785a29e None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:17:20.759042 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217e4c8-3a7d-4b7d-bfe8-9b5dd785a29e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:17:20.759316 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217e4c8-3a7d-4b7d-bfe8-9b5dd785a29e None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:17:20.760621 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 590/1183] 10.222.0.13 () {52 vars in 830 bytes} [Fri Oct 21 21:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:17:20.764537 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08b396-a9a7-4c8c-9bf9-566e7c67b6ff service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:17:20.765733 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ec08b396-a9a7-4c8c-9bf9-566e7c67b6ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:17:20.864427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08b396-a9a7-4c8c-9bf9-566e7c67b6ff service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:17:20.956649 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fc94c-fe7f-4c19-86b5-7ededa2c8137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:17:20.957276 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fc94c-fe7f-4c19-86b5-7ededa2c8137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:17:20.957549 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fc94c-fe7f-4c19-86b5-7ededa2c8137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:17:20.957811 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fc94c-fe7f-4c19-86b5-7ededa2c8137 admin admin] PATH_INFO: `/v3/users/ef619fd1c4824c12aa58bdc525f6b28c` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:17:20.970228 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fc94c-fe7f-4c19-86b5-7ededa2c8137 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef619fd1c4824c12aa58bdc525f6b28c)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:17:20.971821 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: WARNING py.warnings [None req-1e8fc94c-fe7f-4c19-86b5-7ededa2c8137 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 21:17:20.971821 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: warnings.warn(msg) Oct 21 21:17:20.971821 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: Oct 21 21:17:20.973040 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fc94c-fe7f-4c19-86b5-7ededa2c8137 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:17:20.982463 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:17:21.039845 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1e8fc94c-fe7f-4c19-86b5-7ededa2c8137 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef619fd1c4824c12aa58bdc525f6b28c'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:17:21.040503 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1e8fc94c-fe7f-4c19-86b5-7ededa2c8137 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef619fd1c4824c12aa58bdc525f6b28c'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:17:21.062891 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1e8fc94c-fe7f-4c19-86b5-7ededa2c8137 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef619fd1c4824c12aa58bdc525f6b28c'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:17:21.072877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1e8fc94c-fe7f-4c19-86b5-7ededa2c8137 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef619fd1c4824c12aa58bdc525f6b28c'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:17:21.073427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1e8fc94c-fe7f-4c19-86b5-7ededa2c8137 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef619fd1c4824c12aa58bdc525f6b28c'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:17:21.092497 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 594/1184] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:17:20 2022] DELETE /identity/v3/users/ef619fd1c4824c12aa58bdc525f6b28c => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:17:21.097348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef789fd6-aece-4408-afa7-d89c9c2dfd85 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:17:21.097873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef789fd6-aece-4408-afa7-d89c9c2dfd85 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:17:21.097873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef789fd6-aece-4408-afa7-d89c9c2dfd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:17:21.153809 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-ef789fd6-aece-4408-afa7-d89c9c2dfd85 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:17:21.154995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ef789fd6-aece-4408-afa7-d89c9c2dfd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:17:21.229856 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:17:21.243053 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 591/1185] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:17:21.252859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fc94c-fe7f-4c19-86b5-7ededa2c8137 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:17:21.254082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1e8fc94c-fe7f-4c19-86b5-7ededa2c8137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:17:21.355184 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fc94c-fe7f-4c19-86b5-7ededa2c8137 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:17:21.410370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:17:21.430895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb78de-4eea-4a5c-936d-8c3fa9d2ab8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:17:21.431610 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb78de-4eea-4a5c-936d-8c3fa9d2ab8d service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:17:21.431896 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb78de-4eea-4a5c-936d-8c3fa9d2ab8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:17:21.432168 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb78de-4eea-4a5c-936d-8c3fa9d2ab8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:17:21.433763 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-5efb78de-4eea-4a5c-936d-8c3fa9d2ab8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:17:21.543271 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb78de-4eea-4a5c-936d-8c3fa9d2ab8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:17:21.544671 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb78de-4eea-4a5c-936d-8c3fa9d2ab8d service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:17:21.564077 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 595/1186] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:17:21.934777 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef789fd6-aece-4408-afa7-d89c9c2dfd85 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:17:21.936067 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ef789fd6-aece-4408-afa7-d89c9c2dfd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:17:21.947703 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef789fd6-aece-4408-afa7-d89c9c2dfd85 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:17:22.043736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d513bb-3222-4c00-85f3-ee51ab1c7559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:17:22.044387 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d513bb-3222-4c00-85f3-ee51ab1c7559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:17:22.044665 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d513bb-3222-4c00-85f3-ee51ab1c7559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:17:22.044937 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d513bb-3222-4c00-85f3-ee51ab1c7559 admin admin] PATH_INFO: `/v3/projects/0b01e31f642542c28286db03a205dbc0` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:17:22.055174 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d513bb-3222-4c00-85f3-ee51ab1c7559 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b01e31f642542c28286db03a205dbc0)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:17:22.056209 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: WARNING py.warnings [None req-b2d513bb-3222-4c00-85f3-ee51ab1c7559 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 21:17:22.056209 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: warnings.warn(msg) Oct 21 21:17:22.056209 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: Oct 21 21:17:22.057401 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d513bb-3222-4c00-85f3-ee51ab1c7559 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:17:22.116598 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b2d513bb-3222-4c00-85f3-ee51ab1c7559 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b01e31f642542c28286db03a205dbc0', 'request_id': 'req-b2d513bb-3222-4c00-85f3-ee51ab1c7559'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:17:22.129901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b2d513bb-3222-4c00-85f3-ee51ab1c7559 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b01e31f642542c28286db03a205dbc0', 'request_id': 'req-b2d513bb-3222-4c00-85f3-ee51ab1c7559'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:17:22.140553 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b2d513bb-3222-4c00-85f3-ee51ab1c7559 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b01e31f642542c28286db03a205dbc0', 'request_id': 'req-b2d513bb-3222-4c00-85f3-ee51ab1c7559'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:17:22.164519 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b2d513bb-3222-4c00-85f3-ee51ab1c7559 admin admin] The token cache is being invalidate because project 0b01e31f642542c28286db03a205dbc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:17:22.164805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b2d513bb-3222-4c00-85f3-ee51ab1c7559 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:17:22.165328 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b2d513bb-3222-4c00-85f3-ee51ab1c7559 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:17:22.166688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 592/1187] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:17:21 2022] DELETE /identity/v3/projects/0b01e31f642542c28286db03a205dbc0 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:17:31.561742 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=62 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:17:32.162580 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:18:49.008577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e1930-349e-41d2-9657-b1d33c4dbc8a None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:18:49.009161 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e1930-349e-41d2-9657-b1d33c4dbc8a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:18:49.009161 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e1930-349e-41d2-9657-b1d33c4dbc8a None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:18:49.010493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 596/1188] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 21:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:18:49.015389 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900cafb-c81a-4139-9455-b26437d046bc None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:18:49.015854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900cafb-c81a-4139-9455-b26437d046bc None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:18:49.015963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900cafb-c81a-4139-9455-b26437d046bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:18:49.095201 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-8900cafb-c81a-4139-9455-b26437d046bc None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:18:49.096438 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8900cafb-c81a-4139-9455-b26437d046bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:18:49.198224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 593/1189] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 21:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:18:49.353711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e1930-349e-41d2-9657-b1d33c4dbc8a None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:18:49.355843 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-289e1930-349e-41d2-9657-b1d33c4dbc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:18:49.475241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e1930-349e-41d2-9657-b1d33c4dbc8a None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:18:49.480937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85090101-7368-4025-812a-7e1fddf5339f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:18:49.481539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85090101-7368-4025-812a-7e1fddf5339f service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:18:49.481818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85090101-7368-4025-812a-7e1fddf5339f service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:18:49.482089 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85090101-7368-4025-812a-7e1fddf5339f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:18:49.490232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85090101-7368-4025-812a-7e1fddf5339f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:18:49.491844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85090101-7368-4025-812a-7e1fddf5339f service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:18:49.501037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 597/1190] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:18:50.589754 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706c577-3772-4910-8445-5bfe14840ba9 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:18:50.590312 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706c577-3772-4910-8445-5bfe14840ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:18:50.590312 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706c577-3772-4910-8445-5bfe14840ba9 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:18:50.591668 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 594/1191] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 21:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:18:50.598547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5223e-ca3f-4208-9547-7a9293dafdf9 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:18:50.598984 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5223e-ca3f-4208-9547-7a9293dafdf9 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:18:50.599080 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5223e-ca3f-4208-9547-7a9293dafdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:18:50.618610 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-13a5223e-ca3f-4208-9547-7a9293dafdf9 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:18:50.619842 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-13a5223e-ca3f-4208-9547-7a9293dafdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:18:50.627685 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 598/1192] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 21:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:18:50.778815 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706c577-3772-4910-8445-5bfe14840ba9 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:18:50.793655 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706c577-3772-4910-8445-5bfe14840ba9 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:18:50.799119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c70765-727d-4404-ac17-d9eca29248f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:18:50.799762 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c70765-727d-4404-ac17-d9eca29248f5 service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:18:50.800043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c70765-727d-4404-ac17-d9eca29248f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:18:50.800315 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c70765-727d-4404-ac17-d9eca29248f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:18:50.808350 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c70765-727d-4404-ac17-d9eca29248f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:18:50.810255 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c70765-727d-4404-ac17-d9eca29248f5 service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:18:50.821108 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 595/1193] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:18:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:18:50.877472 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfa7e6-8bdb-4432-9b82-dd31af4669c5 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:18:50.877946 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfa7e6-8bdb-4432-9b82-dd31af4669c5 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:18:50.878290 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfa7e6-8bdb-4432-9b82-dd31af4669c5 None None] PATH_INFO: `/v3` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:18:50.879958 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 599/1194] 10.222.0.44 () {54 vars in 925 bytes} [Fri Oct 21 21:18:50 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:18:50.888693 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ead43-f376-4e0f-8ca0-e6013df36092 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:18:50.888996 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ead43-f376-4e0f-8ca0-e6013df36092 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:18:50.889261 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ead43-f376-4e0f-8ca0-e6013df36092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:18:50.962830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-ec6ead43-f376-4e0f-8ca0-e6013df36092 None None] MFA Rules not processed for user `124f2b5b95e8450ea6867f6868b9e810`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:18:50.964144 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ec6ead43-f376-4e0f-8ca0-e6013df36092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:18:50.984507 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:18:51.039814 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 596/1195] 10.222.0.44 () {58 vars in 1015 bytes} [Fri Oct 21 21:18:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:18:59.770144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35399d7-ea1b-43b0-be0d-53dd013f024e None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:18:59.770602 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35399d7-ea1b-43b0-be0d-53dd013f024e None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:18:59.770712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35399d7-ea1b-43b0-be0d-53dd013f024e None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:18:59.772078 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 600/1196] 10.222.0.13 () {54 vars in 961 bytes} [Fri Oct 21 21:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:18:59.777334 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d9055-5d78-4821-943b-1264963e0efe None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:18:59.777759 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d9055-5d78-4821-943b-1264963e0efe None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:18:59.777946 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d9055-5d78-4821-943b-1264963e0efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:18:59.797026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-001d9055-5d78-4821-943b-1264963e0efe None None] MFA Rules not processed for user `124f2b5b95e8450ea6867f6868b9e810`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:18:59.798229 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-001d9055-5d78-4821-943b-1264963e0efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:18:59.807465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 597/1197] 10.222.0.13 () {58 vars in 1058 bytes} [Fri Oct 21 21:18:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:18:59.814951 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35399d7-ea1b-43b0-be0d-53dd013f024e None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:18:59.828622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35399d7-ea1b-43b0-be0d-53dd013f024e None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:18:59.833467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53897e2e-8e07-499c-a5f7-396b182c844c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:18:59.834051 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53897e2e-8e07-499c-a5f7-396b182c844c service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:18:59.834325 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53897e2e-8e07-499c-a5f7-396b182c844c service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:18:59.834593 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53897e2e-8e07-499c-a5f7-396b182c844c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:18:59.842556 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53897e2e-8e07-499c-a5f7-396b182c844c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:18:59.844179 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53897e2e-8e07-499c-a5f7-396b182c844c service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:18:59.854014 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 601/1198] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:18:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:19:02.318018 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab0bb7-64dd-4b3b-9483-2b50b48f01a8 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:02.318611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab0bb7-64dd-4b3b-9483-2b50b48f01a8 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:02.318715 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab0bb7-64dd-4b3b-9483-2b50b48f01a8 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:02.320033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 598/1199] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 21:19:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:19:02.326222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a3ae0-84fd-42e1-a0dc-2e3f973c19c6 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:02.326679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a3ae0-84fd-42e1-a0dc-2e3f973c19c6 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:02.326787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a3ae0-84fd-42e1-a0dc-2e3f973c19c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:02.346548 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-4b5a3ae0-84fd-42e1-a0dc-2e3f973c19c6 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:19:02.347814 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-4b5a3ae0-84fd-42e1-a0dc-2e3f973c19c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:19:02.355593 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 602/1200] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 21:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:19:02.508840 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab0bb7-64dd-4b3b-9483-2b50b48f01a8 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:19:02.524371 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab0bb7-64dd-4b3b-9483-2b50b48f01a8 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:19:02.530048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af6e08-e801-4e8d-9c98-3cda2e3dba1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:19:02.530641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af6e08-e801-4e8d-9c98-3cda2e3dba1d service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:02.530915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af6e08-e801-4e8d-9c98-3cda2e3dba1d service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:02.531180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af6e08-e801-4e8d-9c98-3cda2e3dba1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:02.539660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af6e08-e801-4e8d-9c98-3cda2e3dba1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:19:02.541223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af6e08-e801-4e8d-9c98-3cda2e3dba1d service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:19:02.550252 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 599/1201] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:19:03.626414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513324c6-e7c0-42ba-a338-2f93d69301aa None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:03.626994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513324c6-e7c0-42ba-a338-2f93d69301aa None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:03.626994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513324c6-e7c0-42ba-a338-2f93d69301aa None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:03.628339 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 603/1202] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 21:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:19:03.634995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc4bc6-c467-46a5-8915-6bb707072d36 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:03.635460 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc4bc6-c467-46a5-8915-6bb707072d36 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:03.635608 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc4bc6-c467-46a5-8915-6bb707072d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:03.654748 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-31bc4bc6-c467-46a5-8915-6bb707072d36 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:19:03.655982 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-31bc4bc6-c467-46a5-8915-6bb707072d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:19:03.663992 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 600/1203] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 21:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:19:03.815137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513324c6-e7c0-42ba-a338-2f93d69301aa None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:19:03.829391 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513324c6-e7c0-42ba-a338-2f93d69301aa None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:19:03.834704 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d86bd-a2be-4892-8ec7-d25d6cf35f28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:19:03.835292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d86bd-a2be-4892-8ec7-d25d6cf35f28 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:03.835587 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d86bd-a2be-4892-8ec7-d25d6cf35f28 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:03.835859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d86bd-a2be-4892-8ec7-d25d6cf35f28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:03.843689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d86bd-a2be-4892-8ec7-d25d6cf35f28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:19:03.845251 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d86bd-a2be-4892-8ec7-d25d6cf35f28 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:19:03.855095 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 604/1204] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:19:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:19:07.782517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef046ad-3456-4ac2-ad9b-8bdbaf38d1fc None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:07.783052 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef046ad-3456-4ac2-ad9b-8bdbaf38d1fc None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:07.783052 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef046ad-3456-4ac2-ad9b-8bdbaf38d1fc None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:07.784449 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 601/1205] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 21:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:19:07.790692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed76dcb-b145-4a5d-8a9e-c34c03d36362 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:07.791247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed76dcb-b145-4a5d-8a9e-c34c03d36362 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:07.791247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed76dcb-b145-4a5d-8a9e-c34c03d36362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:07.812224 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-bed76dcb-b145-4a5d-8a9e-c34c03d36362 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:19:07.813456 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-bed76dcb-b145-4a5d-8a9e-c34c03d36362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:19:07.821751 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 605/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 21:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:19:08.112581 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef046ad-3456-4ac2-ad9b-8bdbaf38d1fc None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:19:08.114797 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8ef046ad-3456-4ac2-ad9b-8bdbaf38d1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:19:08.190795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef046ad-3456-4ac2-ad9b-8bdbaf38d1fc None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:19:08.194851 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9a814-9ebf-46db-b887-87ec6604dcd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:19:08.195441 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9a814-9ebf-46db-b887-87ec6604dcd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:08.195747 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9a814-9ebf-46db-b887-87ec6604dcd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:08.196016 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9a814-9ebf-46db-b887-87ec6604dcd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:08.204653 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9a814-9ebf-46db-b887-87ec6604dcd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:19:08.206105 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9a814-9ebf-46db-b887-87ec6604dcd2 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:19:08.215513 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 602/1207] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:19:10.363185 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dd8d0-4074-48e9-8e10-733778ed7f1d None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:10.364300 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dd8d0-4074-48e9-8e10-733778ed7f1d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:10.364300 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dd8d0-4074-48e9-8e10-733778ed7f1d None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:10.365828 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 606/1208] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 21:19:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:19:10.371645 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c2be7-b1b1-4315-a444-318150099519 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:10.372111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c2be7-b1b1-4315-a444-318150099519 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:10.372207 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c2be7-b1b1-4315-a444-318150099519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:10.392170 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-857c2be7-b1b1-4315-a444-318150099519 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:19:10.393342 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-857c2be7-b1b1-4315-a444-318150099519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:19:10.401438 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 603/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 21:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:19:10.689089 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0dd8d0-4074-48e9-8e10-733778ed7f1d None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:19:10.690831 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-4d0dd8d0-4074-48e9-8e10-733778ed7f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:19:10.704069 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0dd8d0-4074-48e9-8e10-733778ed7f1d None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:19:10.709515 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae168c2-8876-46e7-8db7-1eb8733539a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:19:10.710106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae168c2-8876-46e7-8db7-1eb8733539a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:10.710494 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae168c2-8876-46e7-8db7-1eb8733539a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:10.710764 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae168c2-8876-46e7-8db7-1eb8733539a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:10.719484 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae168c2-8876-46e7-8db7-1eb8733539a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:19:10.720948 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae168c2-8876-46e7-8db7-1eb8733539a1 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:19:10.730384 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 607/1210] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:19:14.346028 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae615fd-7e0d-43b4-a271-6e500b628e59 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:14.346615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae615fd-7e0d-43b4-a271-6e500b628e59 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:14.346615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae615fd-7e0d-43b4-a271-6e500b628e59 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:14.348507 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 604/1211] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 21:19:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:19:14.353845 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f1af5-111b-423f-ae99-81293efbc264 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:14.354284 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f1af5-111b-423f-ae99-81293efbc264 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:14.354394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f1af5-111b-423f-ae99-81293efbc264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:14.361861 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=81 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:19:14.375789 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-f24f1af5-111b-423f-ae99-81293efbc264 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:19:14.376967 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-f24f1af5-111b-423f-ae99-81293efbc264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:19:14.386693 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 608/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 21:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:19:14.688805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae615fd-7e0d-43b4-a271-6e500b628e59 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:19:14.690467 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-5ae615fd-7e0d-43b4-a271-6e500b628e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:19:14.755688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:19:14.768909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae615fd-7e0d-43b4-a271-6e500b628e59 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:19:14.772798 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192f1dc-9d0d-455c-9c8a-53e88fda766f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:19:14.773409 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192f1dc-9d0d-455c-9c8a-53e88fda766f service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:14.773688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192f1dc-9d0d-455c-9c8a-53e88fda766f service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:14.773957 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192f1dc-9d0d-455c-9c8a-53e88fda766f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:14.782732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192f1dc-9d0d-455c-9c8a-53e88fda766f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:19:14.784207 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192f1dc-9d0d-455c-9c8a-53e88fda766f service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:19:14.793418 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 605/1213] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:19:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:19:16.930741 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d6bfc-89a6-473a-9020-77c318bab74f None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:16.931293 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d6bfc-89a6-473a-9020-77c318bab74f None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:16.931293 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d6bfc-89a6-473a-9020-77c318bab74f None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:16.932649 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 609/1214] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 21:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:19:16.937980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8baf0-e2ce-4846-aad1-f7c3e28a17f0 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:16.938423 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8baf0-e2ce-4846-aad1-f7c3e28a17f0 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:16.938532 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8baf0-e2ce-4846-aad1-f7c3e28a17f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:16.958227 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-2ed8baf0-e2ce-4846-aad1-f7c3e28a17f0 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:19:16.959419 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2ed8baf0-e2ce-4846-aad1-f7c3e28a17f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:19:16.967278 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 606/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 21:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:19:17.265853 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d6bfc-89a6-473a-9020-77c318bab74f None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:19:17.267533 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-878d6bfc-89a6-473a-9020-77c318bab74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:19:17.281538 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d6bfc-89a6-473a-9020-77c318bab74f None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:19:17.285046 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90af706-8743-4b6a-a6f9-7526afa0a6c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:19:17.285793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90af706-8743-4b6a-a6f9-7526afa0a6c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:19:17.286067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90af706-8743-4b6a-a6f9-7526afa0a6c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:19:17.286333 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90af706-8743-4b6a-a6f9-7526afa0a6c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:19:17.294900 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90af706-8743-4b6a-a6f9-7526afa0a6c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:19:17.296319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90af706-8743-4b6a-a6f9-7526afa0a6c0 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:19:17.305614 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 610/1216] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:19:26.953333 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=11 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:19:27.301346 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=8 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:41.078233 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8f932-7b08-4e9a-9f70-97ac776a254b None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.078912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8f932-7b08-4e9a-9f70-97ac776a254b None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.078912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8f932-7b08-4e9a-9f70-97ac776a254b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.102225 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-5cb8f932-7b08-4e9a-9f70-97ac776a254b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:41.103445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-5cb8f932-7b08-4e9a-9f70-97ac776a254b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:41.118543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 607/1217] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:41.129906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c70118-b280-4eaf-b370-a4aef63e4f01 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.130591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca126162-c8df-41f8-a5bf-90fd9e998522 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.130689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c70118-b280-4eaf-b370-a4aef63e4f01 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.130773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca126162-c8df-41f8-a5bf-90fd9e998522 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.130855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c70118-b280-4eaf-b370-a4aef63e4f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.130937 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca126162-c8df-41f8-a5bf-90fd9e998522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.156189 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-ca126162-c8df-41f8-a5bf-90fd9e998522 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:41.157394 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ca126162-c8df-41f8-a5bf-90fd9e998522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:41.173615 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-93c70118-b280-4eaf-b370-a4aef63e4f01 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:41.174642 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 608/1218] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:41.174848 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-93c70118-b280-4eaf-b370-a4aef63e4f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:41.176453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd59732-6e90-4c56-8117-10a12512d67d None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.176739 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd59732-6e90-4c56-8117-10a12512d67d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.177014 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd59732-6e90-4c56-8117-10a12512d67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.197599 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 611/1219] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:20:41.198470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c70118-b280-4eaf-b370-a4aef63e4f01 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:41.211925 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-8cd59732-6e90-4c56-8117-10a12512d67d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:41.213149 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8cd59732-6e90-4c56-8117-10a12512d67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:41.217252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c70118-b280-4eaf-b370-a4aef63e4f01 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:41.232042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d46f5-7efe-43ca-8918-6dfbee33f73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:41.232670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d46f5-7efe-43ca-8918-6dfbee33f73c admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.232935 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 609/1220] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:41.233062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d46f5-7efe-43ca-8918-6dfbee33f73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.233230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d46f5-7efe-43ca-8918-6dfbee33f73c admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.233853 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d46f5-7efe-43ca-8918-6dfbee33f73c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:20:41.234156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d46f5-7efe-43ca-8918-6dfbee33f73c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:41.234414 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cb458-1af4-4734-b0e8-17caf674cec4 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.234706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cb458-1af4-4734-b0e8-17caf674cec4 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.234981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cb458-1af4-4734-b0e8-17caf674cec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.235159 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: WARNING py.warnings [None req-248d46f5-7efe-43ca-8918-6dfbee33f73c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 21 21:20:41.235159 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: warnings.warn(msg) Oct 21 21:20:41.235159 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: Oct 21 21:20:41.236083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d46f5-7efe-43ca-8918-6dfbee33f73c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:41.260324 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 612/1221] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:20:41.261526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df895fa-991f-469b-9c20-ce5fe19506bc None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.261815 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df895fa-991f-469b-9c20-ce5fe19506bc None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.262090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df895fa-991f-469b-9c20-ce5fe19506bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.273429 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-fa8cb458-1af4-4734-b0e8-17caf674cec4 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:41.274889 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-fa8cb458-1af4-4734-b0e8-17caf674cec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:41.296976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 610/1222] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:41.297664 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cb458-1af4-4734-b0e8-17caf674cec4 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:41.304224 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-9df895fa-991f-469b-9c20-ce5fe19506bc None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:41.305454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9df895fa-991f-469b-9c20-ce5fe19506bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:41.312288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cb458-1af4-4734-b0e8-17caf674cec4 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:41.320370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 613/1223] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:41.321824 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2f4c4-5ee9-4ebd-8231-50ae61754968 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.322115 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2f4c4-5ee9-4ebd-8231-50ae61754968 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.322389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2f4c4-5ee9-4ebd-8231-50ae61754968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.324122 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad16859-e123-439f-bd79-e3d553bdfd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:41.324739 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad16859-e123-439f-bd79-e3d553bdfd80 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.325014 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad16859-e123-439f-bd79-e3d553bdfd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.325283 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad16859-e123-439f-bd79-e3d553bdfd80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.325918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad16859-e123-439f-bd79-e3d553bdfd80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:41.329866 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: WARNING py.warnings [None req-2ad16859-e123-439f-bd79-e3d553bdfd80 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 21:20:41.329866 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: warnings.warn(msg) Oct 21 21:20:41.329866 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: Oct 21 21:20:41.331007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad16859-e123-439f-bd79-e3d553bdfd80 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:41.358155 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-9fb2f4c4-5ee9-4ebd-8231-50ae61754968 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:41.360373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9fb2f4c4-5ee9-4ebd-8231-50ae61754968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:41.372551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 614/1224] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:41.373861 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2f4c4-5ee9-4ebd-8231-50ae61754968 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:41.383195 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 611/1225] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/projects => generated 389 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:41.384771 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594ff2d-d94e-4b28-aa17-aa57573d9f63 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.385129 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594ff2d-d94e-4b28-aa17-aa57573d9f63 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.385520 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2f4c4-5ee9-4ebd-8231-50ae61754968 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:41.385850 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594ff2d-d94e-4b28-aa17-aa57573d9f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.406176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-9594ff2d-d94e-4b28-aa17-aa57573d9f63 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:41.407405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9594ff2d-d94e-4b28-aa17-aa57573d9f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:41.445527 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bc3f6-7ca9-41d5-bc95-8775206e29a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:41.446023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bc3f6-7ca9-41d5-bc95-8775206e29a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.446667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bc3f6-7ca9-41d5-bc95-8775206e29a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.446667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bc3f6-7ca9-41d5-bc95-8775206e29a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.447357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bc3f6-7ca9-41d5-bc95-8775206e29a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:20:41.447696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bc3f6-7ca9-41d5-bc95-8775206e29a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:41.449280 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bc3f6-7ca9-41d5-bc95-8775206e29a9 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:41.464017 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 615/1226] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:20:41.464017 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bc3f6-7ca9-41d5-bc95-8775206e29a9 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:41.469854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 612/1227] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:41.471187 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e35ee-b49c-4c7a-b0c7-60570f07662f None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.471482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e35ee-b49c-4c7a-b0c7-60570f07662f None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.471814 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e35ee-b49c-4c7a-b0c7-60570f07662f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.474808 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bc3f6-7ca9-41d5-bc95-8775206e29a9 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:41.480039 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b051eb3-1ec1-4764-92b6-efb43e6c1219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:41.480658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b051eb3-1ec1-4764-92b6-efb43e6c1219 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.480934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b051eb3-1ec1-4764-92b6-efb43e6c1219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.481240 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b051eb3-1ec1-4764-92b6-efb43e6c1219 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.482011 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b051eb3-1ec1-4764-92b6-efb43e6c1219 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:20:41.482321 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b051eb3-1ec1-4764-92b6-efb43e6c1219 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:41.483969 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b051eb3-1ec1-4764-92b6-efb43e6c1219 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:41.492712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-8d5e35ee-b49c-4c7a-b0c7-60570f07662f None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:41.494067 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8d5e35ee-b49c-4c7a-b0c7-60570f07662f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:41.496863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 616/1228] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:20:41.498301 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1269d-5ed5-411b-8382-8b23506a6f2f None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.498585 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1269d-5ed5-411b-8382-8b23506a6f2f None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.498884 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1269d-5ed5-411b-8382-8b23506a6f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.511798 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 613/1229] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:41.511798 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5d51d-dc2c-43ba-9a38-1ebf90810f48 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.511798 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5d51d-dc2c-43ba-9a38-1ebf90810f48 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.511798 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5d51d-dc2c-43ba-9a38-1ebf90810f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.526812 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-a8e1269d-5ed5-411b-8382-8b23506a6f2f None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:41.528067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a8e1269d-5ed5-411b-8382-8b23506a6f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:41.529351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-8aa5d51d-dc2c-43ba-9a38-1ebf90810f48 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:41.530511 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8aa5d51d-dc2c-43ba-9a38-1ebf90810f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:41.536584 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 617/1230] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:41.537349 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1269d-5ed5-411b-8382-8b23506a6f2f None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:41.542714 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 614/1231] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:41.542714 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd566a-fa4c-45b1-be48-8f90c6dd2643 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.542714 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd566a-fa4c-45b1-be48-8f90c6dd2643 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.542714 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd566a-fa4c-45b1-be48-8f90c6dd2643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.551678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1269d-5ed5-411b-8382-8b23506a6f2f None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:41.556282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9facfc99-76ca-4389-9acb-0ba4ad7c15f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:41.556891 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9facfc99-76ca-4389-9acb-0ba4ad7c15f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.557166 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9facfc99-76ca-4389-9acb-0ba4ad7c15f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.557434 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9facfc99-76ca-4389-9acb-0ba4ad7c15f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.558048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9facfc99-76ca-4389-9acb-0ba4ad7c15f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:41.559048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: WARNING py.warnings [None req-9facfc99-76ca-4389-9acb-0ba4ad7c15f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 21:20:41.559048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: warnings.warn(msg) Oct 21 21:20:41.559048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: Oct 21 21:20:41.560303 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9facfc99-76ca-4389-9acb-0ba4ad7c15f4 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:41.560641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-0fcd566a-fa4c-45b1-be48-8f90c6dd2643 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:41.561832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0fcd566a-fa4c-45b1-be48-8f90c6dd2643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:41.570224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 615/1232] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:41.571027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd566a-fa4c-45b1-be48-8f90c6dd2643 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:41.581170 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd566a-fa4c-45b1-be48-8f90c6dd2643 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:41.584572 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 618/1233] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/users => generated 295 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:41.585883 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98a283-368f-4f15-9e5a-3418e6aa44a0 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.586031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bb062-f864-4cfa-8c76-32516134ddeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:41.586287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98a283-368f-4f15-9e5a-3418e6aa44a0 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.586510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98a283-368f-4f15-9e5a-3418e6aa44a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.586622 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bb062-f864-4cfa-8c76-32516134ddeb admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.586818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bb062-f864-4cfa-8c76-32516134ddeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.587121 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bb062-f864-4cfa-8c76-32516134ddeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.588052 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73bb062-f864-4cfa-8c76-32516134ddeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:41.589817 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73bb062-f864-4cfa-8c76-32516134ddeb admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:41.605713 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-3d98a283-368f-4f15-9e5a-3418e6aa44a0 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:41.607015 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3d98a283-368f-4f15-9e5a-3418e6aa44a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:41.616045 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 619/1234] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:41.616867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98a283-368f-4f15-9e5a-3418e6aa44a0 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:41.620758 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 616/1235] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:41.620758 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb54d8-18f9-4109-8c37-06e7dce2f32a None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.620758 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb54d8-18f9-4109-8c37-06e7dce2f32a None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.620758 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb54d8-18f9-4109-8c37-06e7dce2f32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.626951 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98a283-368f-4f15-9e5a-3418e6aa44a0 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:41.641054 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-87bb54d8-18f9-4109-8c37-06e7dce2f32a None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:41.642250 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-87bb54d8-18f9-4109-8c37-06e7dce2f32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:41.684027 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22e762-1b58-4776-ae18-70567ccbf1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:41.684636 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22e762-1b58-4776-ae18-70567ccbf1fb admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.684919 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22e762-1b58-4776-ae18-70567ccbf1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.685185 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22e762-1b58-4776-ae18-70567ccbf1fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.685810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22e762-1b58-4776-ae18-70567ccbf1fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:41.687728 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22e762-1b58-4776-ae18-70567ccbf1fb admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:41.701259 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 617/1236] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:41.702109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb54d8-18f9-4109-8c37-06e7dce2f32a None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:41.708355 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:41.712852 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb54d8-18f9-4109-8c37-06e7dce2f32a None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:41.718819 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ee7ca-681b-4972-9637-b99617cec60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:41.719432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ee7ca-681b-4972-9637-b99617cec60e admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.719755 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ee7ca-681b-4972-9637-b99617cec60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.720052 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ee7ca-681b-4972-9637-b99617cec60e admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.723493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 620/1237] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/projects => generated 393 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:41.723691 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ee7ca-681b-4972-9637-b99617cec60e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:20:41.723691 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ee7ca-681b-4972-9637-b99617cec60e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:41.723881 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974c834-8792-4cb2-96ed-3b36eddd91bb None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.723881 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974c834-8792-4cb2-96ed-3b36eddd91bb None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.723881 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974c834-8792-4cb2-96ed-3b36eddd91bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.724230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ee7ca-681b-4972-9637-b99617cec60e admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:41.732920 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:41.738724 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 618/1238] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:20:41.740116 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4f50a-fe53-4ffa-87f4-1b144291f755 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.740434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4f50a-fe53-4ffa-87f4-1b144291f755 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.740714 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4f50a-fe53-4ffa-87f4-1b144291f755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.749954 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-6974c834-8792-4cb2-96ed-3b36eddd91bb None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:41.751246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-6974c834-8792-4cb2-96ed-3b36eddd91bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:41.761523 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-29d4f50a-fe53-4ffa-87f4-1b144291f755 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:41.762733 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-29d4f50a-fe53-4ffa-87f4-1b144291f755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:41.813670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 621/1239] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:41.814515 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974c834-8792-4cb2-96ed-3b36eddd91bb None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:41.823502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974c834-8792-4cb2-96ed-3b36eddd91bb None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:41.823809 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 619/1240] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:41.824623 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4f50a-fe53-4ffa-87f4-1b144291f755 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:41.828111 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3166f-3208-4f27-93a9-cc9107033994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:41.828707 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3166f-3208-4f27-93a9-cc9107033994 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.828982 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3166f-3208-4f27-93a9-cc9107033994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.829313 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3166f-3208-4f27-93a9-cc9107033994 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.829876 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3166f-3208-4f27-93a9-cc9107033994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:41.831585 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3166f-3208-4f27-93a9-cc9107033994 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:41.834707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4f50a-fe53-4ffa-87f4-1b144291f755 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:41.840064 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20debf-9938-4964-9c2d-0f466d7e7c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:41.840705 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20debf-9938-4964-9c2d-0f466d7e7c5c admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.841117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20debf-9938-4964-9c2d-0f466d7e7c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.841563 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20debf-9938-4964-9c2d-0f466d7e7c5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.841943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 622/1241] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:41.842414 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b20debf-9938-4964-9c2d-0f466d7e7c5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:20:41.842724 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b20debf-9938-4964-9c2d-0f466d7e7c5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:41.842836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3166f-3208-4f27-93a9-cc9107033994 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:41.844378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b20debf-9938-4964-9c2d-0f466d7e7c5c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:41.852267 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3166f-3208-4f27-93a9-cc9107033994 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:41.863587 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 620/1242] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:20:41.863798 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf8864-3e11-4cd0-b7ad-98977a2910b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:41.863924 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20debf-9938-4964-9c2d-0f466d7e7c5c admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:41.864029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf8864-3e11-4cd0-b7ad-98977a2910b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.864029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf8864-3e11-4cd0-b7ad-98977a2910b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.864029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf8864-3e11-4cd0-b7ad-98977a2910b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.864029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf8864-3e11-4cd0-b7ad-98977a2910b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:41.864029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf8864-3e11-4cd0-b7ad-98977a2910b3 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:41.867353 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20debf-9938-4964-9c2d-0f466d7e7c5c admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:41.872287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffc3ab-b88a-4b65-8d9e-15c61f2532e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:41.872841 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffc3ab-b88a-4b65-8d9e-15c61f2532e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.873147 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffc3ab-b88a-4b65-8d9e-15c61f2532e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.873451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffc3ab-b88a-4b65-8d9e-15c61f2532e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.874048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ffc3ab-b88a-4b65-8d9e-15c61f2532e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:20:41.874374 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ffc3ab-b88a-4b65-8d9e-15c61f2532e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:41.875984 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ffc3ab-b88a-4b65-8d9e-15c61f2532e2 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:41.886637 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 623/1243] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/users => generated 292 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:41.886637 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf8864-3e11-4cd0-b7ad-98977a2910b3 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:41.892904 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 621/1244] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:20:41.894050 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffc3ab-b88a-4b65-8d9e-15c61f2532e2 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:41.894200 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf8864-3e11-4cd0-b7ad-98977a2910b3 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:41.899397 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a5a48-01f2-4a62-9915-a785a2b0d5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:41.900018 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a5a48-01f2-4a62-9915-a785a2b0d5cf admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.900299 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a5a48-01f2-4a62-9915-a785a2b0d5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.900695 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a5a48-01f2-4a62-9915-a785a2b0d5cf admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.901185 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a5a48-01f2-4a62-9915-a785a2b0d5cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:41.903120 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a5a48-01f2-4a62-9915-a785a2b0d5cf admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:41.904681 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffc3ab-b88a-4b65-8d9e-15c61f2532e2 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:41.909292 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64281b76-39a4-4955-9f24-7694b99941e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:41.909914 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64281b76-39a4-4955-9f24-7694b99941e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:41.910231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64281b76-39a4-4955-9f24-7694b99941e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:41.910507 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64281b76-39a4-4955-9f24-7694b99941e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:41.911130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64281b76-39a4-4955-9f24-7694b99941e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:41.913052 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64281b76-39a4-4955-9f24-7694b99941e5 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:41.926111 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 624/1245] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/users => generated 297 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:41.928449 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a5a48-01f2-4a62-9915-a785a2b0d5cf admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:41.939683 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a5a48-01f2-4a62-9915-a785a2b0d5cf admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:41.941476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 622/1246] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:20:41 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:41.942336 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64281b76-39a4-4955-9f24-7694b99941e5 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.001350 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628b144-e28f-46c8-af9e-1dbe96aa4719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.002061 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628b144-e28f-46c8-af9e-1dbe96aa4719 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.002426 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628b144-e28f-46c8-af9e-1dbe96aa4719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.002765 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628b144-e28f-46c8-af9e-1dbe96aa4719 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.003470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628b144-e28f-46c8-af9e-1dbe96aa4719 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:20:42.004378 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628b144-e28f-46c8-af9e-1dbe96aa4719 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.005473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64281b76-39a4-4955-9f24-7694b99941e5 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.005972 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628b144-e28f-46c8-af9e-1dbe96aa4719 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.010178 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1757e-d14b-406b-8288-017bbd1d8de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.010776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1757e-d14b-406b-8288-017bbd1d8de2 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.011065 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1757e-d14b-406b-8288-017bbd1d8de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.011328 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1757e-d14b-406b-8288-017bbd1d8de2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.012056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1757e-d14b-406b-8288-017bbd1d8de2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:20:42.012369 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1757e-d14b-406b-8288-017bbd1d8de2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.014123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1757e-d14b-406b-8288-017bbd1d8de2 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.024656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 625/1247] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:20:42.024656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628b144-e28f-46c8-af9e-1dbe96aa4719 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.026388 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 623/1248] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:20:42.027227 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1757e-d14b-406b-8288-017bbd1d8de2 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.036012 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628b144-e28f-46c8-af9e-1dbe96aa4719 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.038714 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1757e-d14b-406b-8288-017bbd1d8de2 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.040666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db1666-4b74-4fd2-83c6-30a96c66ebb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.041262 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db1666-4b74-4fd2-83c6-30a96c66ebb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.041540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db1666-4b74-4fd2-83c6-30a96c66ebb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.041866 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db1666-4b74-4fd2-83c6-30a96c66ebb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.042442 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db1666-4b74-4fd2-83c6-30a96c66ebb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.043367 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54750f19-cdb3-443b-9423-ab267c54f1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.044035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54750f19-cdb3-443b-9423-ab267c54f1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.044148 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db1666-4b74-4fd2-83c6-30a96c66ebb9 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.044307 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54750f19-cdb3-443b-9423-ab267c54f1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.044526 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54750f19-cdb3-443b-9423-ab267c54f1a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.045141 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54750f19-cdb3-443b-9423-ab267c54f1a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.047025 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54750f19-cdb3-443b-9423-ab267c54f1a1 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.054408 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 626/1249] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:42.058309 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db1666-4b74-4fd2-83c6-30a96c66ebb9 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.068859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db1666-4b74-4fd2-83c6-30a96c66ebb9 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.073888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefaa9f-e724-4f2f-a583-a79815819c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.074484 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefaa9f-e724-4f2f-a583-a79815819c60 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.074763 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefaa9f-e724-4f2f-a583-a79815819c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.078577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefaa9f-e724-4f2f-a583-a79815819c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.078725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 624/1250] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:20:42 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:42.079206 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefaa9f-e724-4f2f-a583-a79815819c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.079313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54750f19-cdb3-443b-9423-ab267c54f1a1 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.079414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefaa9f-e724-4f2f-a583-a79815819c60 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.092683 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 627/1251] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:42.092683 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefaa9f-e724-4f2f-a583-a79815819c60 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.094060 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:42.103747 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefaa9f-e724-4f2f-a583-a79815819c60 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.131655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:42.141511 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54750f19-cdb3-443b-9423-ab267c54f1a1 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.147432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bfa09-4e5b-49c6-b9a5-ee55d3ea1f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.148081 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bfa09-4e5b-49c6-b9a5-ee55d3ea1f24 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.148783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bfa09-4e5b-49c6-b9a5-ee55d3ea1f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.149063 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bfa09-4e5b-49c6-b9a5-ee55d3ea1f24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.149694 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617bfa09-4e5b-49c6-b9a5-ee55d3ea1f24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.152315 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617bfa09-4e5b-49c6-b9a5-ee55d3ea1f24 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.167504 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f43d4f-7193-452d-9f2c-42f4348e2693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.168160 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f43d4f-7193-452d-9f2c-42f4348e2693 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.168436 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f43d4f-7193-452d-9f2c-42f4348e2693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.168710 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f43d4f-7193-452d-9f2c-42f4348e2693 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.169349 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f43d4f-7193-452d-9f2c-42f4348e2693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.171155 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f43d4f-7193-452d-9f2c-42f4348e2693 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.179801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 625/1252] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:20:42 2022] POST /identity/v3/projects => generated 357 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:42.180605 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bfa09-4e5b-49c6-b9a5-ee55d3ea1f24 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.184572 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 628/1253] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:42.184572 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f43d4f-7193-452d-9f2c-42f4348e2693 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.194672 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f43d4f-7193-452d-9f2c-42f4348e2693 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.243945 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bfa09-4e5b-49c6-b9a5-ee55d3ea1f24 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.248955 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f01ed9-9049-4cac-8ac5-652d9719121a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.249562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f01ed9-9049-4cac-8ac5-652d9719121a admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.249840 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f01ed9-9049-4cac-8ac5-652d9719121a admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.250111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f01ed9-9049-4cac-8ac5-652d9719121a admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.250733 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f01ed9-9049-4cac-8ac5-652d9719121a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.252713 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f01ed9-9049-4cac-8ac5-652d9719121a admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.252922 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0c432-3746-41e2-b247-c7090b8b2a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.253576 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0c432-3746-41e2-b247-c7090b8b2a54 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.254359 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0c432-3746-41e2-b247-c7090b8b2a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.254629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0c432-3746-41e2-b247-c7090b8b2a54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.255257 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0c432-3746-41e2-b247-c7090b8b2a54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.257175 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0c432-3746-41e2-b247-c7090b8b2a54 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.278525 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 626/1254] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:20:42 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:42.278525 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f01ed9-9049-4cac-8ac5-652d9719121a admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.286304 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f01ed9-9049-4cac-8ac5-652d9719121a admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.290983 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 629/1255] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:20:42 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:42.290983 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0c432-3746-41e2-b247-c7090b8b2a54 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.345197 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb896d0-12ab-4a6c-bf19-390b5dbf414c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.345712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb896d0-12ab-4a6c-bf19-390b5dbf414c admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.345989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb896d0-12ab-4a6c-bf19-390b5dbf414c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.346256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb896d0-12ab-4a6c-bf19-390b5dbf414c admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.346879 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb896d0-12ab-4a6c-bf19-390b5dbf414c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.348855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb896d0-12ab-4a6c-bf19-390b5dbf414c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.352911 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0c432-3746-41e2-b247-c7090b8b2a54 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.357809 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b2e50-5632-4f24-bebc-432290e4e7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.358459 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b2e50-5632-4f24-bebc-432290e4e7fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.358734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b2e50-5632-4f24-bebc-432290e4e7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.359005 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b2e50-5632-4f24-bebc-432290e4e7fd admin admin] PATH_INFO: `/v3/projects/62d044836c10479ca7c1975d4a64cbe8/users/64e9cccb41a04aef83ab417783d82fcb/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.374776 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b2e50-5632-4f24-bebc-432290e4e7fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=62d044836c10479ca7c1975d4a64cbe8, user_id=64e9cccb41a04aef83ab417783d82fcb, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.376017 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 627/1256] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:20:42 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:42.376826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb896d0-12ab-4a6c-bf19-390b5dbf414c admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.381366 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b2e50-5632-4f24-bebc-432290e4e7fd admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.390228 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:42.396127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 630/1257] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:20:42 2022] PUT /identity/v3/projects/62d044836c10479ca7c1975d4a64cbe8/users/64e9cccb41a04aef83ab417783d82fcb/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:20:42.397137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b2e50-5632-4f24-bebc-432290e4e7fd admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.431439 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:42.440862 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb896d0-12ab-4a6c-bf19-390b5dbf414c admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.461467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b2e50-5632-4f24-bebc-432290e4e7fd admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.466110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4900a9f-1b05-44e2-bae5-12f0ae98d787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.466713 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4900a9f-1b05-44e2-bae5-12f0ae98d787 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.466989 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4900a9f-1b05-44e2-bae5-12f0ae98d787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.467259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4900a9f-1b05-44e2-bae5-12f0ae98d787 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.467920 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4900a9f-1b05-44e2-bae5-12f0ae98d787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.469542 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4900a9f-1b05-44e2-bae5-12f0ae98d787 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.484666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 631/1258] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:42.484666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4900a9f-1b05-44e2-bae5-12f0ae98d787 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.495261 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4900a9f-1b05-44e2-bae5-12f0ae98d787 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.496983 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24062e-6158-49bc-babe-4264bc249624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.497597 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24062e-6158-49bc-babe-4264bc249624 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.497882 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24062e-6158-49bc-babe-4264bc249624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.498151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24062e-6158-49bc-babe-4264bc249624 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.498775 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24062e-6158-49bc-babe-4264bc249624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.500004 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e0de4-f053-4183-9a6f-954660298d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.500622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e0de4-f053-4183-9a6f-954660298d5b admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.500879 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24062e-6158-49bc-babe-4264bc249624 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.500985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e0de4-f053-4183-9a6f-954660298d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.501151 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e0de4-f053-4183-9a6f-954660298d5b admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.501762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e0de4-f053-4183-9a6f-954660298d5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.503727 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e0de4-f053-4183-9a6f-954660298d5b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.524062 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 628/1259] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:20:42 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:42.524062 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24062e-6158-49bc-babe-4264bc249624 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.532821 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 632/1260] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:20:42 2022] POST /identity/v3/users => generated 279 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:42.532821 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e0de4-f053-4183-9a6f-954660298d5b admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.537796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24062e-6158-49bc-babe-4264bc249624 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.541475 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e0de4-f053-4183-9a6f-954660298d5b admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.542555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45aa89-9ee4-4988-ac54-43a5452ac876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.543162 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45aa89-9ee4-4988-ac54-43a5452ac876 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.543562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45aa89-9ee4-4988-ac54-43a5452ac876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.543867 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45aa89-9ee4-4988-ac54-43a5452ac876 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.544501 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45aa89-9ee4-4988-ac54-43a5452ac876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.546103 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45aa89-9ee4-4988-ac54-43a5452ac876 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.546244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45194607-8994-4905-9dab-7d02f961392a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.546867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45194607-8994-4905-9dab-7d02f961392a admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.547146 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45194607-8994-4905-9dab-7d02f961392a admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.547656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45194607-8994-4905-9dab-7d02f961392a admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.548289 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45194607-8994-4905-9dab-7d02f961392a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.549843 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45194607-8994-4905-9dab-7d02f961392a admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.556565 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 629/1261] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:42.557437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45aa89-9ee4-4988-ac54-43a5452ac876 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.565926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 633/1262] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:42.565926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45194607-8994-4905-9dab-7d02f961392a admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.570748 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45aa89-9ee4-4988-ac54-43a5452ac876 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.575498 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059da23-49a3-4906-b762-5a15bbbda6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.576133 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059da23-49a3-4906-b762-5a15bbbda6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.576413 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059da23-49a3-4906-b762-5a15bbbda6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.576608 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45194607-8994-4905-9dab-7d02f961392a admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.576706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059da23-49a3-4906-b762-5a15bbbda6e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.577290 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059da23-49a3-4906-b762-5a15bbbda6e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.579201 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059da23-49a3-4906-b762-5a15bbbda6e5 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.581015 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd475bed-34a1-43b9-b3b7-ba42bbf7ad2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.581619 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd475bed-34a1-43b9-b3b7-ba42bbf7ad2d admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.581943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd475bed-34a1-43b9-b3b7-ba42bbf7ad2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.582210 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd475bed-34a1-43b9-b3b7-ba42bbf7ad2d admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.582827 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd475bed-34a1-43b9-b3b7-ba42bbf7ad2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.584726 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd475bed-34a1-43b9-b3b7-ba42bbf7ad2d admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.602502 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 630/1263] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:20:42 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:42.604965 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cea1c5-07c5-4df5-acc5-16c250fd3b0b None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.605246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cea1c5-07c5-4df5-acc5-16c250fd3b0b None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.605584 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cea1c5-07c5-4df5-acc5-16c250fd3b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.607290 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 634/1264] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:20:42 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:42.608144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd475bed-34a1-43b9-b3b7-ba42bbf7ad2d admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.621758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd475bed-34a1-43b9-b3b7-ba42bbf7ad2d admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.626435 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b88cd-1423-435b-b75b-995129d27025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.627085 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b88cd-1423-435b-b75b-995129d27025 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.627213 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-89cea1c5-07c5-4df5-acc5-16c250fd3b0b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:42.627366 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b88cd-1423-435b-b75b-995129d27025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.627667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b88cd-1423-435b-b75b-995129d27025 admin admin] PATH_INFO: `/v3/projects/57ba110632704f1f94a2621a0d2afbbe/users/c25df0b554494b6085081a1e18a5a1d8/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.628364 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-89cea1c5-07c5-4df5-acc5-16c250fd3b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:42.636432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 631/1265] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:42.637223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cea1c5-07c5-4df5-acc5-16c250fd3b0b None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.643130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b88cd-1423-435b-b75b-995129d27025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57ba110632704f1f94a2621a0d2afbbe, user_id=c25df0b554494b6085081a1e18a5a1d8, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.646693 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b88cd-1423-435b-b75b-995129d27025 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.647242 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cea1c5-07c5-4df5-acc5-16c250fd3b0b None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.652010 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f8a92-bbeb-4fab-9967-2c2c5f5ea414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.652745 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f8a92-bbeb-4fab-9967-2c2c5f5ea414 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.652991 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f8a92-bbeb-4fab-9967-2c2c5f5ea414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.653289 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f8a92-bbeb-4fab-9967-2c2c5f5ea414 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.653919 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f8a92-bbeb-4fab-9967-2c2c5f5ea414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.655729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f8a92-bbeb-4fab-9967-2c2c5f5ea414 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.660507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 635/1266] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:20:42 2022] PUT /identity/v3/projects/57ba110632704f1f94a2621a0d2afbbe/users/c25df0b554494b6085081a1e18a5a1d8/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:20:42.669057 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b88cd-1423-435b-b75b-995129d27025 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.669172 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 632/1267] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:42.669172 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f8a92-bbeb-4fab-9967-2c2c5f5ea414 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.678926 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f8a92-bbeb-4fab-9967-2c2c5f5ea414 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.740110 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571a4c0-b65b-476f-8045-12723e3d9de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.740260 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b88cd-1423-435b-b75b-995129d27025 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.740779 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571a4c0-b65b-476f-8045-12723e3d9de2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.741056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571a4c0-b65b-476f-8045-12723e3d9de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.741351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571a4c0-b65b-476f-8045-12723e3d9de2 admin admin] PATH_INFO: `/v3/projects/e00d05e0e86e4553ac0a953949538ce1/users/e550c842cdbd4da396045c50a6d38866/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.744902 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82142ece-0bc7-4c5f-b684-77f99e59fd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.745500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82142ece-0bc7-4c5f-b684-77f99e59fd7c admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.745769 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82142ece-0bc7-4c5f-b684-77f99e59fd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.746032 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82142ece-0bc7-4c5f-b684-77f99e59fd7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.746664 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82142ece-0bc7-4c5f-b684-77f99e59fd7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.748297 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82142ece-0bc7-4c5f-b684-77f99e59fd7c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.750073 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:42.758587 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571a4c0-b65b-476f-8045-12723e3d9de2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e00d05e0e86e4553ac0a953949538ce1, user_id=e550c842cdbd4da396045c50a6d38866, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.759093 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 636/1268] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:42.759960 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82142ece-0bc7-4c5f-b684-77f99e59fd7c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.761427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571a4c0-b65b-476f-8045-12723e3d9de2 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.770719 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82142ece-0bc7-4c5f-b684-77f99e59fd7c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.780069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 633/1269] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:20:42 2022] PUT /identity/v3/projects/e00d05e0e86e4553ac0a953949538ce1/users/e550c842cdbd4da396045c50a6d38866/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:20:42.780069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571a4c0-b65b-476f-8045-12723e3d9de2 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.824757 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:42.833038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2da26-d534-4101-8572-2d1e263f36df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.833879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2da26-d534-4101-8572-2d1e263f36df admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.834342 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2da26-d534-4101-8572-2d1e263f36df admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.834682 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2da26-d534-4101-8572-2d1e263f36df admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.835715 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2da26-d534-4101-8572-2d1e263f36df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.837227 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2da26-d534-4101-8572-2d1e263f36df admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.841572 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571a4c0-b65b-476f-8045-12723e3d9de2 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.847293 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 637/1270] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:42.848002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9e58a-5047-489a-bb52-8ef3e7e34208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.848379 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9e58a-5047-489a-bb52-8ef3e7e34208 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.848479 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2da26-d534-4101-8572-2d1e263f36df admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.848659 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9e58a-5047-489a-bb52-8ef3e7e34208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.848931 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9e58a-5047-489a-bb52-8ef3e7e34208 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.849560 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9e58a-5047-489a-bb52-8ef3e7e34208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.851228 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9e58a-5047-489a-bb52-8ef3e7e34208 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.860391 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2da26-d534-4101-8572-2d1e263f36df admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.861336 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 634/1271] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:42.862133 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9e58a-5047-489a-bb52-8ef3e7e34208 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.866004 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c9ec9-80a3-4771-bb86-aec245cfaa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.866623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c9ec9-80a3-4771-bb86-aec245cfaa0d admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.866896 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c9ec9-80a3-4771-bb86-aec245cfaa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.867162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c9ec9-80a3-4771-bb86-aec245cfaa0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.867828 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c9ec9-80a3-4771-bb86-aec245cfaa0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.869403 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c9ec9-80a3-4771-bb86-aec245cfaa0d admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.884918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 638/1272] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:42.884918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c9ec9-80a3-4771-bb86-aec245cfaa0d admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.895940 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c9ec9-80a3-4771-bb86-aec245cfaa0d admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.900717 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30159c9a-49da-4d98-9371-147f6a932be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.901299 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30159c9a-49da-4d98-9371-147f6a932be5 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.901571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30159c9a-49da-4d98-9371-147f6a932be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.901836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30159c9a-49da-4d98-9371-147f6a932be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.902453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30159c9a-49da-4d98-9371-147f6a932be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.904054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30159c9a-49da-4d98-9371-147f6a932be5 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.914499 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 639/1273] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:42.915847 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b98b9-012b-4322-8782-de32944675e3 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.916131 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b98b9-012b-4322-8782-de32944675e3 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.916401 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b98b9-012b-4322-8782-de32944675e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.920275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9e58a-5047-489a-bb52-8ef3e7e34208 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.923858 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919300d-4c7d-46ab-b648-c5887483f56e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.924476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919300d-4c7d-46ab-b648-c5887483f56e service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.924755 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919300d-4c7d-46ab-b648-c5887483f56e service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.925037 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919300d-4c7d-46ab-b648-c5887483f56e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.934663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919300d-4c7d-46ab-b648-c5887483f56e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.936550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919300d-4c7d-46ab-b648-c5887483f56e service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.944365 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-1b3b98b9-012b-4322-8782-de32944675e3 None None] MFA Rules not processed for user `c25df0b554494b6085081a1e18a5a1d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:42.945600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1b3b98b9-012b-4322-8782-de32944675e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:42.948033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 635/1274] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:20:42.948894 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919300d-4c7d-46ab-b648-c5887483f56e service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:42.958370 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919300d-4c7d-46ab-b648-c5887483f56e service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:42.962987 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a29a3d-51c9-4875-8078-65672b556f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:42.963525 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a29a3d-51c9-4875-8078-65672b556f40 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.963842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a29a3d-51c9-4875-8078-65672b556f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.964126 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a29a3d-51c9-4875-8078-65672b556f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.964745 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a29a3d-51c9-4875-8078-65672b556f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:42.966341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a29a3d-51c9-4875-8078-65672b556f40 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:42.978021 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 636/1275] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:42.979416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bb385-3bd5-4a3c-9324-433cd091ba8b None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:42.979726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bb385-3bd5-4a3c-9324-433cd091ba8b None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:42.980001 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bb385-3bd5-4a3c-9324-433cd091ba8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:42.999148 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-9f4bb385-3bd5-4a3c-9324-433cd091ba8b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:43.000947 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9f4bb385-3bd5-4a3c-9324-433cd091ba8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:43.012663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 637/1276] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:43.012663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4bb385-3bd5-4a3c-9324-433cd091ba8b None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:43.020379 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4bb385-3bd5-4a3c-9324-433cd091ba8b None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:43.026142 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee56780-dd43-43e3-a675-63e45c7ca90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:43.026801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee56780-dd43-43e3-a675-63e45c7ca90d admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.027079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee56780-dd43-43e3-a675-63e45c7ca90d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.027356 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee56780-dd43-43e3-a675-63e45c7ca90d admin admin] PATH_INFO: `/v3/projects/cdd4b4b0f51940ee829562303eca9ce9/users/00374760bfa648e48dedae8aadb61c8f/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.032880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 640/1277] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:43.038398 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b98b9-012b-4322-8782-de32944675e3 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:43.043606 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee56780-dd43-43e3-a675-63e45c7ca90d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdd4b4b0f51940ee829562303eca9ce9, user_id=00374760bfa648e48dedae8aadb61c8f, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:43.046441 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee56780-dd43-43e3-a675-63e45c7ca90d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:43.053743 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b98b9-012b-4322-8782-de32944675e3 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:43.063886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1231fb-7bbf-4c1e-9956-a7a7c05147e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:43.065347 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1231fb-7bbf-4c1e-9956-a7a7c05147e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.065347 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1231fb-7bbf-4c1e-9956-a7a7c05147e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.067489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1231fb-7bbf-4c1e-9956-a7a7c05147e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.068182 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1231fb-7bbf-4c1e-9956-a7a7c05147e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:43.070204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 638/1278] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:20:43 2022] PUT /identity/v3/projects/cdd4b4b0f51940ee829562303eca9ce9/users/00374760bfa648e48dedae8aadb61c8f/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:20:43.070895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee56780-dd43-43e3-a675-63e45c7ca90d admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:43.071717 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1231fb-7bbf-4c1e-9956-a7a7c05147e9 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:43.084902 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 641/1279] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:43.085751 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1231fb-7bbf-4c1e-9956-a7a7c05147e9 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:43.098332 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1231fb-7bbf-4c1e-9956-a7a7c05147e9 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:43.130273 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:43.151271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee56780-dd43-43e3-a675-63e45c7ca90d admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:43.158091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f282c54-5794-4c78-9bf0-dcdb522540e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:43.158918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f282c54-5794-4c78-9bf0-dcdb522540e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.159196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f282c54-5794-4c78-9bf0-dcdb522540e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.159465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f282c54-5794-4c78-9bf0-dcdb522540e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.160168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f282c54-5794-4c78-9bf0-dcdb522540e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:43.161899 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f282c54-5794-4c78-9bf0-dcdb522540e1 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:43.168070 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6668d3-aef8-42ed-b822-b6d19b11ca36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:43.168799 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6668d3-aef8-42ed-b822-b6d19b11ca36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.169027 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6668d3-aef8-42ed-b822-b6d19b11ca36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.169302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6668d3-aef8-42ed-b822-b6d19b11ca36 admin admin] PATH_INFO: `/v3/projects/8122be51915c4754b86202ec79b4d197/users/dc5388b90d0847739330a095f4d5f106/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.173655 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 639/1280] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:43.177380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f282c54-5794-4c78-9bf0-dcdb522540e1 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:43.187190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6668d3-aef8-42ed-b822-b6d19b11ca36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8122be51915c4754b86202ec79b4d197, user_id=dc5388b90d0847739330a095f4d5f106, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:43.187376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f282c54-5794-4c78-9bf0-dcdb522540e1 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:43.190310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6668d3-aef8-42ed-b822-b6d19b11ca36 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:43.193362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cce395-20b7-4589-8a09-0c23a5ef1eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:43.194133 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cce395-20b7-4589-8a09-0c23a5ef1eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.194543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cce395-20b7-4589-8a09-0c23a5ef1eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.194862 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cce395-20b7-4589-8a09-0c23a5ef1eaf admin admin] PATH_INFO: `/v3/projects/4100f18f980e45618e20917e6323eb2f/users/3f22e1859b1c4952a0f77cb5f7e2e586/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.208886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 642/1281] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:20:43 2022] PUT /identity/v3/projects/8122be51915c4754b86202ec79b4d197/users/dc5388b90d0847739330a095f4d5f106/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:20:43.209837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6668d3-aef8-42ed-b822-b6d19b11ca36 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:43.214815 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cce395-20b7-4589-8a09-0c23a5ef1eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4100f18f980e45618e20917e6323eb2f, user_id=3f22e1859b1c4952a0f77cb5f7e2e586, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:43.218526 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cce395-20b7-4589-8a09-0c23a5ef1eaf admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:43.236417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 640/1282] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:20:43 2022] PUT /identity/v3/projects/4100f18f980e45618e20917e6323eb2f/users/3f22e1859b1c4952a0f77cb5f7e2e586/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:20:43.248620 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d60f6-b61f-4bb6-9bfc-8ca73f6dbebe None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.248916 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d60f6-b61f-4bb6-9bfc-8ca73f6dbebe None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.249182 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d60f6-b61f-4bb6-9bfc-8ca73f6dbebe None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.250705 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 641/1283] 10.222.0.13 () {54 vars in 961 bytes} [Fri Oct 21 21:20:43 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:20:43.251718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:43.251936 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe603a-c6e3-42c7-991d-7a1f5240244f None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.252795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe603a-c6e3-42c7-991d-7a1f5240244f None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.253072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe603a-c6e3-42c7-991d-7a1f5240244f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.275792 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-d0fe603a-c6e3-42c7-991d-7a1f5240244f None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:43.277096 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d0fe603a-c6e3-42c7-991d-7a1f5240244f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:43.287477 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6668d3-aef8-42ed-b822-b6d19b11ca36 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:43.343463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 642/1284] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:43.343463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe603a-c6e3-42c7-991d-7a1f5240244f None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:43.350890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6349f-e91c-4ca8-a291-cf80562f54ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:43.351189 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe603a-c6e3-42c7-991d-7a1f5240244f None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:43.351709 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6349f-e91c-4ca8-a291-cf80562f54ea service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.352283 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6349f-e91c-4ca8-a291-cf80562f54ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.352751 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6349f-e91c-4ca8-a291-cf80562f54ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.355957 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfc2ad-1093-43cf-aa05-be1eb1b04739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:43.356613 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfc2ad-1093-43cf-aa05-be1eb1b04739 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.356896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfc2ad-1093-43cf-aa05-be1eb1b04739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.357171 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfc2ad-1093-43cf-aa05-be1eb1b04739 admin admin] PATH_INFO: `/v3/projects/58272cc82d764177aa008a9c06abbb17/users/eafdf1ea23b448a1acd4f96c1a0b0aec/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.364943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6349f-e91c-4ca8-a291-cf80562f54ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:43.366453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6349f-e91c-4ca8-a291-cf80562f54ea service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:43.374228 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfc2ad-1093-43cf-aa05-be1eb1b04739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58272cc82d764177aa008a9c06abbb17, user_id=eafdf1ea23b448a1acd4f96c1a0b0aec, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:43.376976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfc2ad-1093-43cf-aa05-be1eb1b04739 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:43.378314 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 643/1285] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:20:43.379479 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6349f-e91c-4ca8-a291-cf80562f54ea service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:43.389852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6349f-e91c-4ca8-a291-cf80562f54ea service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:43.398999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 643/1286] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:20:43 2022] PUT /identity/v3/projects/58272cc82d764177aa008a9c06abbb17/users/eafdf1ea23b448a1acd4f96c1a0b0aec/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:20:43.398999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23b78a-808b-48a4-8411-606ac5b4cee1 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.398999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23b78a-808b-48a4-8411-606ac5b4cee1 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.398999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23b78a-808b-48a4-8411-606ac5b4cee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.419924 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-1a23b78a-808b-48a4-8411-606ac5b4cee1 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:43.421300 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-1a23b78a-808b-48a4-8411-606ac5b4cee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:43.479730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0e7de-dc7d-45cb-b602-be29b21effa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:43.480502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0e7de-dc7d-45cb-b602-be29b21effa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.480795 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0e7de-dc7d-45cb-b602-be29b21effa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.482749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0e7de-dc7d-45cb-b602-be29b21effa1 admin admin] PATH_INFO: `/v3/projects/b1c0e61355eb4a97b7d19595c466ad3b/users/c6e5113e504a4348b74d89a48ba50c17/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.501732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 644/1287] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:43.504256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178bd2f-57d8-45ec-8fdd-0c2a45cf9166 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.504782 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178bd2f-57d8-45ec-8fdd-0c2a45cf9166 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.505532 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178bd2f-57d8-45ec-8fdd-0c2a45cf9166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.509209 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0e7de-dc7d-45cb-b602-be29b21effa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c0e61355eb4a97b7d19595c466ad3b, user_id=c6e5113e504a4348b74d89a48ba50c17, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:43.512154 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0e7de-dc7d-45cb-b602-be29b21effa1 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:43.538826 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 644/1288] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:20:43 2022] PUT /identity/v3/projects/b1c0e61355eb4a97b7d19595c466ad3b/users/c6e5113e504a4348b74d89a48ba50c17/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:20:43.542167 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6aed1-356d-455d-8b9a-877c66b8a842 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.542699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6aed1-356d-455d-8b9a-877c66b8a842 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.542923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6aed1-356d-455d-8b9a-877c66b8a842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.544163 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-0178bd2f-57d8-45ec-8fdd-0c2a45cf9166 None None] MFA Rules not processed for user `3f22e1859b1c4952a0f77cb5f7e2e586`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:43.545644 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0178bd2f-57d8-45ec-8fdd-0c2a45cf9166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:43.579797 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:43.601761 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-61b6aed1-356d-455d-8b9a-877c66b8a842 None None] MFA Rules not processed for user `12dad9f016a740b98538722e1b723823`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:43.605379 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-61b6aed1-356d-455d-8b9a-877c66b8a842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:43.656937 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 645/1289] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:43.658396 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178bd2f-57d8-45ec-8fdd-0c2a45cf9166 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:43.683709 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:43.706055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 645/1290] 10.222.0.13 () {58 vars in 1058 bytes} [Fri Oct 21 21:20:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:43.707527 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddb0fc-5e6e-4dca-ba52-3780f19233ba None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.707836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddb0fc-5e6e-4dca-ba52-3780f19233ba None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.708107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddb0fc-5e6e-4dca-ba52-3780f19233ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.748474 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178bd2f-57d8-45ec-8fdd-0c2a45cf9166 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:43.752070 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-89ddb0fc-5e6e-4dca-ba52-3780f19233ba None None] MFA Rules not processed for user `eafdf1ea23b448a1acd4f96c1a0b0aec`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:43.753319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-89ddb0fc-5e6e-4dca-ba52-3780f19233ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:43.754252 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7442cc-270a-468e-a1c0-f223b55d7ff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:43.755053 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7442cc-270a-468e-a1c0-f223b55d7ff3 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.755379 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7442cc-270a-468e-a1c0-f223b55d7ff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.755730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7442cc-270a-468e-a1c0-f223b55d7ff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.839693 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7442cc-270a-468e-a1c0-f223b55d7ff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:43.841181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7442cc-270a-468e-a1c0-f223b55d7ff3 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:43.856010 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 646/1291] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:20:43.856963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7442cc-270a-468e-a1c0-f223b55d7ff3 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:43.866931 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7442cc-270a-468e-a1c0-f223b55d7ff3 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:43.871453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d8cd3-d43c-471f-8fd7-e8a2912a6869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:43.872314 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d8cd3-d43c-471f-8fd7-e8a2912a6869 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.872769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d8cd3-d43c-471f-8fd7-e8a2912a6869 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.873735 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d8cd3-d43c-471f-8fd7-e8a2912a6869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.885724 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d8cd3-d43c-471f-8fd7-e8a2912a6869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:43.888928 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d8cd3-d43c-471f-8fd7-e8a2912a6869 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:43.890626 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 646/1292] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:43.892577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd4f50-d4a5-4db5-9194-ea1e699687b5 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.892870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd4f50-d4a5-4db5-9194-ea1e699687b5 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.893152 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd4f50-d4a5-4db5-9194-ea1e699687b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.911830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 647/1293] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:20:43.912744 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d8cd3-d43c-471f-8fd7-e8a2912a6869 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:43.930492 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d8cd3-d43c-471f-8fd7-e8a2912a6869 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:43.941759 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e185cf3-bd1a-4cbe-a008-34c0e4e2649b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:43.942555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e185cf3-bd1a-4cbe-a008-34c0e4e2649b admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.942900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e185cf3-bd1a-4cbe-a008-34c0e4e2649b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.944521 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e185cf3-bd1a-4cbe-a008-34c0e4e2649b admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.944521 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e185cf3-bd1a-4cbe-a008-34c0e4e2649b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:43.946456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e185cf3-bd1a-4cbe-a008-34c0e4e2649b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:43.958885 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-80cd4f50-d4a5-4db5-9194-ea1e699687b5 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:43.963272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-80cd4f50-d4a5-4db5-9194-ea1e699687b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:43.979101 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 647/1294] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:43.980162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd4f50-d4a5-4db5-9194-ea1e699687b5 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:43.993413 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 648/1295] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:20:43 2022] POST /identity/v3/projects => generated 357 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:43.993413 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee5b3b-4554-482f-a9f6-5b80eca79179 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:43.993413 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee5b3b-4554-482f-a9f6-5b80eca79179 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:43.993413 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee5b3b-4554-482f-a9f6-5b80eca79179 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:43.994868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 649/1296] 10.222.0.13 () {54 vars in 961 bytes} [Fri Oct 21 21:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:20:43.998454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd4f50-d4a5-4db5-9194-ea1e699687b5 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:44.001141 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee5b3b-4554-482f-a9f6-5b80eca79179 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:44.064444 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:44.099213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e082195-b503-4ff4-92f4-f4f7e7777a8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:44.100201 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e082195-b503-4ff4-92f4-f4f7e7777a8d service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:44.100624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e082195-b503-4ff4-92f4-f4f7e7777a8d service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:44.100997 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e082195-b503-4ff4-92f4-f4f7e7777a8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:44.101196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee5b3b-4554-482f-a9f6-5b80eca79179 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:44.106451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c143a-a218-4078-b01a-13f0ea85cb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:44.107172 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c143a-a218-4078-b01a-13f0ea85cb72 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:44.107456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c143a-a218-4078-b01a-13f0ea85cb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:44.108337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c143a-a218-4078-b01a-13f0ea85cb72 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:44.108990 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c143a-a218-4078-b01a-13f0ea85cb72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:44.111238 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c143a-a218-4078-b01a-13f0ea85cb72 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:44.150759 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 650/1297] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:20:43 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:44.153029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010d68d-f6dc-495c-94b7-06da10096fa2 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:44.155789 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010d68d-f6dc-495c-94b7-06da10096fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:44.156434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010d68d-f6dc-495c-94b7-06da10096fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:44.198373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e082195-b503-4ff4-92f4-f4f7e7777a8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:44.200278 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e082195-b503-4ff4-92f4-f4f7e7777a8d service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:44.219277 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-d010d68d-f6dc-495c-94b7-06da10096fa2 None None] MFA Rules not processed for user `12dad9f016a740b98538722e1b723823`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:44.219860 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 648/1298] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:20:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:20:44.221263 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e082195-b503-4ff4-92f4-f4f7e7777a8d service nova] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:44.222010 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d010d68d-f6dc-495c-94b7-06da10096fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:44.256782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:44.259696 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 651/1299] 10.222.0.13 () {58 vars in 1058 bytes} [Fri Oct 21 21:20:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:44.263550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010d68d-f6dc-495c-94b7-06da10096fa2 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:44.288614 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010d68d-f6dc-495c-94b7-06da10096fa2 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:44.303343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfba7e2-be85-422a-b736-ef9ce2bb9fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:44.303941 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfba7e2-be85-422a-b736-ef9ce2bb9fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:44.304240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfba7e2-be85-422a-b736-ef9ce2bb9fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:44.304524 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfba7e2-be85-422a-b736-ef9ce2bb9fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:44.305182 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfba7e2-be85-422a-b736-ef9ce2bb9fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:44.311900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfba7e2-be85-422a-b736-ef9ce2bb9fa8 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:44.338742 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 652/1300] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:44.341720 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfba7e2-be85-422a-b736-ef9ce2bb9fa8 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:44.356136 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e082195-b503-4ff4-92f4-f4f7e7777a8d service nova] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:44.361253 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f146d-66d7-4c0f-8b2e-b8eabb6b88a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:44.362707 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f146d-66d7-4c0f-8b2e-b8eabb6b88a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:44.363117 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f146d-66d7-4c0f-8b2e-b8eabb6b88a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:44.363458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f146d-66d7-4c0f-8b2e-b8eabb6b88a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:44.375222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f146d-66d7-4c0f-8b2e-b8eabb6b88a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:44.375497 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfba7e2-be85-422a-b736-ef9ce2bb9fa8 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:44.376845 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f146d-66d7-4c0f-8b2e-b8eabb6b88a1 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:44.390617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b8c33-8d40-4b39-98b0-87cccc4ecc1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:44.390953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 649/1301] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:20:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:20:44.391542 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b8c33-8d40-4b39-98b0-87cccc4ecc1e service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:44.391847 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f146d-66d7-4c0f-8b2e-b8eabb6b88a1 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:44.391963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b8c33-8d40-4b39-98b0-87cccc4ecc1e service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:44.392120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b8c33-8d40-4b39-98b0-87cccc4ecc1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:44.421755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f146d-66d7-4c0f-8b2e-b8eabb6b88a1 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:44.428052 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d40c33-6c4b-48d2-af0a-a56207c674cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:44.429312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d40c33-6c4b-48d2-af0a-a56207c674cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:44.429312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d40c33-6c4b-48d2-af0a-a56207c674cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:44.429505 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d40c33-6c4b-48d2-af0a-a56207c674cd admin admin] PATH_INFO: `/v3/projects/39f1e345c0a3416789513a519c804544/users/51775c01d12b487d9015dcb6fd7265a8/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:44.482398 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d40c33-6c4b-48d2-af0a-a56207c674cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=39f1e345c0a3416789513a519c804544, user_id=51775c01d12b487d9015dcb6fd7265a8, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:44.485483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d40c33-6c4b-48d2-af0a-a56207c674cd admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:44.514765 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 650/1302] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:20:44 2022] PUT /identity/v3/projects/39f1e345c0a3416789513a519c804544/users/51775c01d12b487d9015dcb6fd7265a8/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:20:44.517330 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d40c33-6c4b-48d2-af0a-a56207c674cd admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:44.539007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b8c33-8d40-4b39-98b0-87cccc4ecc1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:44.541528 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b8c33-8d40-4b39-98b0-87cccc4ecc1e service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:44.567306 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 653/1303] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:20:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:20:44.571446 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb232b00-f77a-4eb6-a028-9c6e098ea4e8 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:44.571852 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb232b00-f77a-4eb6-a028-9c6e098ea4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:44.572735 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb232b00-f77a-4eb6-a028-9c6e098ea4e8 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:44.578176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 654/1304] 10.222.0.13 () {54 vars in 912 bytes} [Fri Oct 21 21:20:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:20:44.579473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c163925-49af-4912-b3e7-e96ffdb44af9 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:44.579795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c163925-49af-4912-b3e7-e96ffdb44af9 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:44.580439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c163925-49af-4912-b3e7-e96ffdb44af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:44.605035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-9c163925-49af-4912-b3e7-e96ffdb44af9 None None] MFA Rules not processed for user `12dad9f016a740b98538722e1b723823`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:44.606389 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9c163925-49af-4912-b3e7-e96ffdb44af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:44.684728 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d40c33-6c4b-48d2-af0a-a56207c674cd admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:44.697173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c9aa7-3018-4b11-b6fe-7de10d01b108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:44.699196 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c9aa7-3018-4b11-b6fe-7de10d01b108 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:44.700180 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c9aa7-3018-4b11-b6fe-7de10d01b108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:44.702497 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c9aa7-3018-4b11-b6fe-7de10d01b108 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:44.703510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c9aa7-3018-4b11-b6fe-7de10d01b108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:44.706110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c9aa7-3018-4b11-b6fe-7de10d01b108 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:44.716324 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 655/1305] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Oct 21 21:20:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:44.726247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c163925-49af-4912-b3e7-e96ffdb44af9 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:44.743175 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 651/1306] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:44.744484 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c9aa7-3018-4b11-b6fe-7de10d01b108 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:44.767376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c9aa7-3018-4b11-b6fe-7de10d01b108 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:44.781497 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65a85a-4a6b-4d9f-9404-2f5d56f78834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:44.783847 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65a85a-4a6b-4d9f-9404-2f5d56f78834 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:44.784222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65a85a-4a6b-4d9f-9404-2f5d56f78834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:44.784601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65a85a-4a6b-4d9f-9404-2f5d56f78834 admin admin] PATH_INFO: `/v3/projects/39f1e345c0a3416789513a519c804544/users/51775c01d12b487d9015dcb6fd7265a8/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:44.791088 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65a85a-4a6b-4d9f-9404-2f5d56f78834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39f1e345c0a3416789513a519c804544, user_id=51775c01d12b487d9015dcb6fd7265a8, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:44.796791 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65a85a-4a6b-4d9f-9404-2f5d56f78834 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:44.820725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:44.830627 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 652/1307] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:20:44 2022] PUT /identity/v3/projects/39f1e345c0a3416789513a519c804544/users/51775c01d12b487d9015dcb6fd7265a8/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:20:44.832823 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cea23-2252-48d7-9f77-a2e4bc8b200d None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:44.833331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cea23-2252-48d7-9f77-a2e4bc8b200d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:44.833773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cea23-2252-48d7-9f77-a2e4bc8b200d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:44.861041 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c163925-49af-4912-b3e7-e96ffdb44af9 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:44.902017 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-3b8cea23-2252-48d7-9f77-a2e4bc8b200d None None] MFA Rules not processed for user `51775c01d12b487d9015dcb6fd7265a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:44.907869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3b8cea23-2252-48d7-9f77-a2e4bc8b200d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:44.964780 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5273d-51d8-4946-9a36-7b2684ce8609 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:44.965562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5273d-51d8-4946-9a36-7b2684ce8609 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:44.965962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5273d-51d8-4946-9a36-7b2684ce8609 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:44.966338 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5273d-51d8-4946-9a36-7b2684ce8609 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:45.034093 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:45.049372 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 653/1308] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:44 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:45.057488 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cea23-2252-48d7-9f77-a2e4bc8b200d None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:45.074775 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cea23-2252-48d7-9f77-a2e4bc8b200d None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:45.084165 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a85df3-b753-4d60-b700-f7a1553672e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:45.084946 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a85df3-b753-4d60-b700-f7a1553672e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:45.085345 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a85df3-b753-4d60-b700-f7a1553672e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:45.085721 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a85df3-b753-4d60-b700-f7a1553672e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:45.110752 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a85df3-b753-4d60-b700-f7a1553672e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:45.114823 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a85df3-b753-4d60-b700-f7a1553672e8 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:45.123967 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5273d-51d8-4946-9a36-7b2684ce8609 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:45.125909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5273d-51d8-4946-9a36-7b2684ce8609 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:45.141894 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 654/1309] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:20:45 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:20:45.143142 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234045e-b794-4616-8d18-6788db1206cc None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:45.143958 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234045e-b794-4616-8d18-6788db1206cc None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:45.145434 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234045e-b794-4616-8d18-6788db1206cc None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:45.149001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 655/1310] 10.222.0.13 () {54 vars in 912 bytes} [Fri Oct 21 21:20:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:20:45.152655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d696b1-9cc6-415a-b445-c1876d7b5587 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:45.153055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d696b1-9cc6-415a-b445-c1876d7b5587 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:45.153436 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d696b1-9cc6-415a-b445-c1876d7b5587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:45.168132 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 656/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:20:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:20:45.207076 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-d3d696b1-9cc6-415a-b445-c1876d7b5587 None None] MFA Rules not processed for user `12dad9f016a740b98538722e1b723823`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:45.210294 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d3d696b1-9cc6-415a-b445-c1876d7b5587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:45.235107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 656/1312] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Oct 21 21:20:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:45.261884 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5273d-51d8-4946-9a36-7b2684ce8609 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:45.275253 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5273d-51d8-4946-9a36-7b2684ce8609 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:45.285681 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b882d-f511-4f04-ac98-e93bfda717ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:45.286317 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b882d-f511-4f04-ac98-e93bfda717ec service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:45.289967 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b882d-f511-4f04-ac98-e93bfda717ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:45.290266 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b882d-f511-4f04-ac98-e93bfda717ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:45.318817 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b882d-f511-4f04-ac98-e93bfda717ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:45.323145 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b882d-f511-4f04-ac98-e93bfda717ec service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:45.337024 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 657/1313] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:20:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:20:46.135344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d696b1-9cc6-415a-b445-c1876d7b5587 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:46.149407 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d696b1-9cc6-415a-b445-c1876d7b5587 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:46.156118 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36310c9d-07a5-4f30-8968-2c9ad059c681 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:46.157772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36310c9d-07a5-4f30-8968-2c9ad059c681 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:46.159463 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36310c9d-07a5-4f30-8968-2c9ad059c681 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:46.159987 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36310c9d-07a5-4f30-8968-2c9ad059c681 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:46.378864 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36310c9d-07a5-4f30-8968-2c9ad059c681 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:46.380467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36310c9d-07a5-4f30-8968-2c9ad059c681 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:46.419707 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 657/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:20:46 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:20:50.454853 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b882d-f511-4f04-ac98-e93bfda717ec service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:50.648258 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b882d-f511-4f04-ac98-e93bfda717ec service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:50.661656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f6de0-cedc-4e30-aa81-5121eb83ad42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:50.663498 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f6de0-cedc-4e30-aa81-5121eb83ad42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:50.664295 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f6de0-cedc-4e30-aa81-5121eb83ad42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:50.665263 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f6de0-cedc-4e30-aa81-5121eb83ad42 admin admin] PATH_INFO: `/v3/users/eafdf1ea23b448a1acd4f96c1a0b0aec` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:50.668604 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f6de0-cedc-4e30-aa81-5121eb83ad42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eafdf1ea23b448a1acd4f96c1a0b0aec)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:50.674887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: WARNING py.warnings [None req-d29f6de0-cedc-4e30-aa81-5121eb83ad42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 21:20:50.674887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: warnings.warn(msg) Oct 21 21:20:50.674887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: Oct 21 21:20:50.680682 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f6de0-cedc-4e30-aa81-5121eb83ad42 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:50.803828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:50.864657 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d29f6de0-cedc-4e30-aa81-5121eb83ad42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eafdf1ea23b448a1acd4f96c1a0b0aec'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:20:50.900153 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d29f6de0-cedc-4e30-aa81-5121eb83ad42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eafdf1ea23b448a1acd4f96c1a0b0aec'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:20:50.915323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d29f6de0-cedc-4e30-aa81-5121eb83ad42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eafdf1ea23b448a1acd4f96c1a0b0aec'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:20:50.920956 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d29f6de0-cedc-4e30-aa81-5121eb83ad42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eafdf1ea23b448a1acd4f96c1a0b0aec'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:20:50.936600 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d29f6de0-cedc-4e30-aa81-5121eb83ad42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eafdf1ea23b448a1acd4f96c1a0b0aec'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:20:50.964976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 658/1315] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:20:50 2022] DELETE /identity/v3/users/eafdf1ea23b448a1acd4f96c1a0b0aec => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:20:50.969499 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92238b80-242f-494f-8731-828e71404db0 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:50.971694 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92238b80-242f-494f-8731-828e71404db0 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:50.972177 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92238b80-242f-494f-8731-828e71404db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:51.053436 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-92238b80-242f-494f-8731-828e71404db0 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:51.055034 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-92238b80-242f-494f-8731-828e71404db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:51.220232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 658/1316] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:51.339210 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f6de0-cedc-4e30-aa81-5121eb83ad42 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:51.341007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d29f6de0-cedc-4e30-aa81-5121eb83ad42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:51.436852 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f6de0-cedc-4e30-aa81-5121eb83ad42 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:51.441591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54845bdc-9643-4d15-a2c4-9a4e0c1872ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:51.442344 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54845bdc-9643-4d15-a2c4-9a4e0c1872ee service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:51.442813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54845bdc-9643-4d15-a2c4-9a4e0c1872ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:51.443182 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54845bdc-9643-4d15-a2c4-9a4e0c1872ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:51.455697 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54845bdc-9643-4d15-a2c4-9a4e0c1872ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:51.458036 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54845bdc-9643-4d15-a2c4-9a4e0c1872ee service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:51.472323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 659/1317] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:20:52.727689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92238b80-242f-494f-8731-828e71404db0 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:52.730767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-92238b80-242f-494f-8731-828e71404db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:52.754489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92238b80-242f-494f-8731-828e71404db0 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:52.765193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda07223-d0fc-4fe3-b710-8705bc104002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:52.766332 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda07223-d0fc-4fe3-b710-8705bc104002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:52.766839 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda07223-d0fc-4fe3-b710-8705bc104002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:52.769700 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda07223-d0fc-4fe3-b710-8705bc104002 admin admin] PATH_INFO: `/v3/projects/58272cc82d764177aa008a9c06abbb17` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:52.771297 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda07223-d0fc-4fe3-b710-8705bc104002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58272cc82d764177aa008a9c06abbb17)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:52.774655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: WARNING py.warnings [None req-bda07223-d0fc-4fe3-b710-8705bc104002 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 21:20:52.774655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: warnings.warn(msg) Oct 21 21:20:52.774655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: Oct 21 21:20:52.776517 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda07223-d0fc-4fe3-b710-8705bc104002 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:52.891429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bda07223-d0fc-4fe3-b710-8705bc104002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58272cc82d764177aa008a9c06abbb17', 'request_id': 'req-bda07223-d0fc-4fe3-b710-8705bc104002'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:20:52.909299 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bda07223-d0fc-4fe3-b710-8705bc104002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58272cc82d764177aa008a9c06abbb17', 'request_id': 'req-bda07223-d0fc-4fe3-b710-8705bc104002'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:20:52.919554 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bda07223-d0fc-4fe3-b710-8705bc104002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58272cc82d764177aa008a9c06abbb17', 'request_id': 'req-bda07223-d0fc-4fe3-b710-8705bc104002'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:20:52.955259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:52.966514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bda07223-d0fc-4fe3-b710-8705bc104002 admin admin] The token cache is being invalidate because project 58272cc82d764177aa008a9c06abbb17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:20:52.966514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bda07223-d0fc-4fe3-b710-8705bc104002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:20:52.968237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bda07223-d0fc-4fe3-b710-8705bc104002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:20:52.970062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 659/1318] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:20:52 2022] DELETE /identity/v3/projects/58272cc82d764177aa008a9c06abbb17 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:20:52.975872 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418279f-655e-49f4-9742-d464e3713bf1 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:52.977797 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418279f-655e-49f4-9742-d464e3713bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:52.978308 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418279f-655e-49f4-9742-d464e3713bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:53.078311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-4418279f-655e-49f4-9742-d464e3713bf1 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:53.079613 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-4418279f-655e-49f4-9742-d464e3713bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:53.253266 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 660/1319] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:53.262010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fabe3-a621-4d0d-b368-d70216575ad6 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:53.262547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fabe3-a621-4d0d-b368-d70216575ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:53.262978 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fabe3-a621-4d0d-b368-d70216575ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:53.325657 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-fa0fabe3-a621-4d0d-b368-d70216575ad6 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:53.326937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-fa0fabe3-a621-4d0d-b368-d70216575ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:53.367036 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 660/1320] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:53.371205 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418279f-655e-49f4-9742-d464e3713bf1 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:53.387815 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418279f-655e-49f4-9742-d464e3713bf1 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:53.415695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0773b-fbb0-49d2-b35f-956d030f5678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:53.416364 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0773b-fbb0-49d2-b35f-956d030f5678 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:53.416675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0773b-fbb0-49d2-b35f-956d030f5678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:53.416901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0773b-fbb0-49d2-b35f-956d030f5678 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:53.417565 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0773b-fbb0-49d2-b35f-956d030f5678 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:20:53.417869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0773b-fbb0-49d2-b35f-956d030f5678 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:53.419543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0773b-fbb0-49d2-b35f-956d030f5678 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:53.461926 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 661/1321] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:20:53.466827 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0fabe3-a621-4d0d-b368-d70216575ad6 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:53.480684 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0fabe3-a621-4d0d-b368-d70216575ad6 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:53.496489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b288aac-8bab-4ca4-9a88-d322585e6a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:53.498180 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b288aac-8bab-4ca4-9a88-d322585e6a5c admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:53.499033 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b288aac-8bab-4ca4-9a88-d322585e6a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:53.500103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b288aac-8bab-4ca4-9a88-d322585e6a5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:53.506013 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b288aac-8bab-4ca4-9a88-d322585e6a5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:53.508342 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b288aac-8bab-4ca4-9a88-d322585e6a5c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:53.589503 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 661/1322] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:20:53 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:53.589709 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0773b-fbb0-49d2-b35f-956d030f5678 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:53.608241 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0773b-fbb0-49d2-b35f-956d030f5678 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:53.681522 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:53.754232 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77579af0-4732-4045-a8bf-f534dc232c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:53.754879 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77579af0-4732-4045-a8bf-f534dc232c93 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:53.755160 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77579af0-4732-4045-a8bf-f534dc232c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:53.755434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77579af0-4732-4045-a8bf-f534dc232c93 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:53.756258 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77579af0-4732-4045-a8bf-f534dc232c93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:53.758255 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77579af0-4732-4045-a8bf-f534dc232c93 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:53.807955 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 662/1323] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:20:53 2022] POST /identity/v3/users => generated 284 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:20:53.812019 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b288aac-8bab-4ca4-9a88-d322585e6a5c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:53.832365 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b288aac-8bab-4ca4-9a88-d322585e6a5c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:53.855411 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4593ea6-1da9-46f4-8170-e6c6d032de7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:53.864594 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4593ea6-1da9-46f4-8170-e6c6d032de7d admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:53.864891 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4593ea6-1da9-46f4-8170-e6c6d032de7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:53.867100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4593ea6-1da9-46f4-8170-e6c6d032de7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:53.867838 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4593ea6-1da9-46f4-8170-e6c6d032de7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:53.869491 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4593ea6-1da9-46f4-8170-e6c6d032de7d admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:53.896145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 662/1324] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:53.900827 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77579af0-4732-4045-a8bf-f534dc232c93 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:53.915087 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77579af0-4732-4045-a8bf-f534dc232c93 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:53.932381 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb338b-625e-4690-84e7-8a6ac79564ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:53.932956 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb338b-625e-4690-84e7-8a6ac79564ab admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:53.933237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb338b-625e-4690-84e7-8a6ac79564ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:53.933506 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb338b-625e-4690-84e7-8a6ac79564ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:53.934285 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb338b-625e-4690-84e7-8a6ac79564ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:53.961560 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb338b-625e-4690-84e7-8a6ac79564ab admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:53.995035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 663/1325] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:53.999637 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4593ea6-1da9-46f4-8170-e6c6d032de7d admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:54.017960 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4593ea6-1da9-46f4-8170-e6c6d032de7d admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:54.039918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577d774-41f3-4a56-adbf-c13fb1a06fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:54.040622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577d774-41f3-4a56-adbf-c13fb1a06fbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:54.040975 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577d774-41f3-4a56-adbf-c13fb1a06fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:54.041422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577d774-41f3-4a56-adbf-c13fb1a06fbe admin admin] PATH_INFO: `/v3/projects/3cf8471a7b094168b256a5cfe555fec8/users/88856db0aab648d9a9fe8cac873051e9/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:54.080502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577d774-41f3-4a56-adbf-c13fb1a06fbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cf8471a7b094168b256a5cfe555fec8, user_id=88856db0aab648d9a9fe8cac873051e9, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:54.083650 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577d774-41f3-4a56-adbf-c13fb1a06fbe admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:54.145654 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 663/1326] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:20:53 2022] PUT /identity/v3/projects/3cf8471a7b094168b256a5cfe555fec8/users/88856db0aab648d9a9fe8cac873051e9/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:20:54.165206 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae276fc-e371-49a0-9870-ae8f9541c385 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:54.168311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae276fc-e371-49a0-9870-ae8f9541c385 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:54.168758 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae276fc-e371-49a0-9870-ae8f9541c385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:54.256726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-5ae276fc-e371-49a0-9870-ae8f9541c385 None None] MFA Rules not processed for user `88856db0aab648d9a9fe8cac873051e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:54.258002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-5ae276fc-e371-49a0-9870-ae8f9541c385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:54.413554 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 664/1327] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:54.414988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577d774-41f3-4a56-adbf-c13fb1a06fbe admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:54.438315 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcf9e7-ff19-4616-96a9-43ee9c0cd8ff None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:54.438588 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcf9e7-ff19-4616-96a9-43ee9c0cd8ff None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:54.438859 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcf9e7-ff19-4616-96a9-43ee9c0cd8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:54.515628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-b8bcf9e7-ff19-4616-96a9-43ee9c0cd8ff None None] MFA Rules not processed for user `e550c842cdbd4da396045c50a6d38866`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:54.516878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b8bcf9e7-ff19-4616-96a9-43ee9c0cd8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:54.594598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577d774-41f3-4a56-adbf-c13fb1a06fbe admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:54.609961 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f3efe-a462-4db8-91ec-28486defe736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:54.610456 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f3efe-a462-4db8-91ec-28486defe736 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:54.610737 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f3efe-a462-4db8-91ec-28486defe736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:54.611011 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f3efe-a462-4db8-91ec-28486defe736 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:54.611701 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f3efe-a462-4db8-91ec-28486defe736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:54.617713 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f3efe-a462-4db8-91ec-28486defe736 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:54.676614 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 664/1328] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:20:54 2022] POST /identity/v3/projects => generated 367 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:54.680075 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 665/1329] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:20:54.681075 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcf9e7-ff19-4616-96a9-43ee9c0cd8ff None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:54.690730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37309738-20f2-47f5-803f-3d9226b71aa4 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:54.694866 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37309738-20f2-47f5-803f-3d9226b71aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:54.695343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37309738-20f2-47f5-803f-3d9226b71aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:54.696699 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcf9e7-ff19-4616-96a9-43ee9c0cd8ff None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:54.742108 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:54.778001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-37309738-20f2-47f5-803f-3d9226b71aa4 None None] MFA Rules not processed for user `e550c842cdbd4da396045c50a6d38866`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:54.796269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-37309738-20f2-47f5-803f-3d9226b71aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:54.841742 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f675478-9b5f-4973-8fd4-722a11668efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:54.844038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f675478-9b5f-4973-8fd4-722a11668efb admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:54.844381 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f675478-9b5f-4973-8fd4-722a11668efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:54.845533 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f675478-9b5f-4973-8fd4-722a11668efb admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:54.846234 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f675478-9b5f-4973-8fd4-722a11668efb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:54.861969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f675478-9b5f-4973-8fd4-722a11668efb admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:54.921202 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 666/1330] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:20:54 2022] POST /identity/v3/users => generated 290 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:20:54.922370 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f675478-9b5f-4973-8fd4-722a11668efb admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:54.958281 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f675478-9b5f-4973-8fd4-722a11668efb admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:54.978259 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116e6c7-86e2-4b97-bf5c-e9553ce787d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:54.978903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116e6c7-86e2-4b97-bf5c-e9553ce787d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:54.979170 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116e6c7-86e2-4b97-bf5c-e9553ce787d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:54.979438 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116e6c7-86e2-4b97-bf5c-e9553ce787d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:54.980117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116e6c7-86e2-4b97-bf5c-e9553ce787d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:54.990330 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116e6c7-86e2-4b97-bf5c-e9553ce787d9 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:54.999129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 665/1331] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:55.010708 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37309738-20f2-47f5-803f-3d9226b71aa4 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:55.015401 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-37309738-20f2-47f5-803f-3d9226b71aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:55.031338 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 667/1332] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:55.038798 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116e6c7-86e2-4b97-bf5c-e9553ce787d9 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:55.082490 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116e6c7-86e2-4b97-bf5c-e9553ce787d9 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:55.095310 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229cb18-7196-4642-a949-978fe3143d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:55.096018 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229cb18-7196-4642-a949-978fe3143d56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:55.096296 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229cb18-7196-4642-a949-978fe3143d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:55.096569 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229cb18-7196-4642-a949-978fe3143d56 admin admin] PATH_INFO: `/v3/projects/db65f62e17404a0abd6046bfbc9f8b05/users/8fd019cd5127453f9c7200a007b1d305/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:55.122089 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:55.141615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229cb18-7196-4642-a949-978fe3143d56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db65f62e17404a0abd6046bfbc9f8b05, user_id=8fd019cd5127453f9c7200a007b1d305, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:55.144873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229cb18-7196-4642-a949-978fe3143d56 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:55.216047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 668/1333] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:20:55 2022] PUT /identity/v3/projects/db65f62e17404a0abd6046bfbc9f8b05/users/8fd019cd5127453f9c7200a007b1d305/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:20:55.217303 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229cb18-7196-4642-a949-978fe3143d56 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:55.297371 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37309738-20f2-47f5-803f-3d9226b71aa4 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:55.372071 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229cb18-7196-4642-a949-978fe3143d56 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:55.391950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e809a9-bf16-45cf-b641-97d08004b6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:55.392874 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e809a9-bf16-45cf-b641-97d08004b6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:55.393330 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e809a9-bf16-45cf-b641-97d08004b6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:55.393790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e809a9-bf16-45cf-b641-97d08004b6d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:55.395140 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e809a9-bf16-45cf-b641-97d08004b6d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:55.398844 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e809a9-bf16-45cf-b641-97d08004b6d7 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:55.426187 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 669/1334] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:20:55.432475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e809a9-bf16-45cf-b641-97d08004b6d7 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:55.445988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be164376-7aeb-44a2-a66a-99d2aa45f7f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:55.447036 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be164376-7aeb-44a2-a66a-99d2aa45f7f5 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:55.447393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be164376-7aeb-44a2-a66a-99d2aa45f7f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:55.447799 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be164376-7aeb-44a2-a66a-99d2aa45f7f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:55.453534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e809a9-bf16-45cf-b641-97d08004b6d7 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:55.469416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413284d-0d05-4f05-83de-be03dc05b65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:55.470404 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413284d-0d05-4f05-83de-be03dc05b65b admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:55.470858 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413284d-0d05-4f05-83de-be03dc05b65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:55.471301 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413284d-0d05-4f05-83de-be03dc05b65b admin admin] PATH_INFO: `/v3/projects/db65f62e17404a0abd6046bfbc9f8b05/users/8fd019cd5127453f9c7200a007b1d305/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:55.482569 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413284d-0d05-4f05-83de-be03dc05b65b admin admin] RBAC: Authorizing `identity:create_grant(project_id=db65f62e17404a0abd6046bfbc9f8b05, user_id=8fd019cd5127453f9c7200a007b1d305, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:55.486274 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413284d-0d05-4f05-83de-be03dc05b65b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:55.542335 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 670/1335] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:20:55 2022] PUT /identity/v3/projects/db65f62e17404a0abd6046bfbc9f8b05/users/8fd019cd5127453f9c7200a007b1d305/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:20:55.544288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de096d-31de-40fb-b136-4f3262bdec23 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:55.544583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de096d-31de-40fb-b136-4f3262bdec23 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:55.544866 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de096d-31de-40fb-b136-4f3262bdec23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:55.583219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be164376-7aeb-44a2-a66a-99d2aa45f7f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:55.587741 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be164376-7aeb-44a2-a66a-99d2aa45f7f5 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:55.614353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 666/1336] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:20:55 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:20:55.661676 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-f7de096d-31de-40fb-b136-4f3262bdec23 None None] MFA Rules not processed for user `8fd019cd5127453f9c7200a007b1d305`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:55.662950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f7de096d-31de-40fb-b136-4f3262bdec23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:55.921069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:55.972471 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 671/1337] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:20:55.980778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be164376-7aeb-44a2-a66a-99d2aa45f7f5 service nova] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:55.982208 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-be164376-7aeb-44a2-a66a-99d2aa45f7f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:56.138509 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be164376-7aeb-44a2-a66a-99d2aa45f7f5 service nova] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:56.168631 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:56.268388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf950b9-517b-498e-9ef2-8533891ee632 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:56.269201 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf950b9-517b-498e-9ef2-8533891ee632 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:56.269458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf950b9-517b-498e-9ef2-8533891ee632 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:56.269736 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf950b9-517b-498e-9ef2-8533891ee632 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:56.294645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf950b9-517b-498e-9ef2-8533891ee632 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:56.296113 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf950b9-517b-498e-9ef2-8533891ee632 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:56.321668 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 667/1338] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:20:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:20:56.707146 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cec381-1e28-414e-b144-84d9a79308e7 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:56.707759 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cec381-1e28-414e-b144-84d9a79308e7 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:56.707759 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cec381-1e28-414e-b144-84d9a79308e7 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:56.715934 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 672/1339] 10.222.0.13 () {54 vars in 912 bytes} [Fri Oct 21 21:20:56 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:20:56.726101 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bf515-d4f6-4d93-8b4a-e2e1c7e96828 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:56.726101 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bf515-d4f6-4d93-8b4a-e2e1c7e96828 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:56.726101 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bf515-d4f6-4d93-8b4a-e2e1c7e96828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:56.844253 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-2c6bf515-d4f6-4d93-8b4a-e2e1c7e96828 None None] MFA Rules not processed for user `d34dc4e7c5474b6b96977019328d8890`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:56.847066 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2c6bf515-d4f6-4d93-8b4a-e2e1c7e96828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:57.036394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 668/1340] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Oct 21 21:20:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:20:57.276992 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cec381-1e28-414e-b144-84d9a79308e7 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:57.278951 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-a9cec381-1e28-414e-b144-84d9a79308e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:57.309679 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cec381-1e28-414e-b144-84d9a79308e7 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:57.318812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80881d2-92a3-4d37-acdc-9be7b857277d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:57.319447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80881d2-92a3-4d37-acdc-9be7b857277d service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:57.319782 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80881d2-92a3-4d37-acdc-9be7b857277d service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:57.320031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80881d2-92a3-4d37-acdc-9be7b857277d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:57.337069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80881d2-92a3-4d37-acdc-9be7b857277d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:57.338565 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80881d2-92a3-4d37-acdc-9be7b857277d service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:57.361855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 673/1341] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:20:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:20:58.573299 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bf515-d4f6-4d93-8b4a-e2e1c7e96828 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:58.577117 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2c6bf515-d4f6-4d93-8b4a-e2e1c7e96828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:58.783397 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bf515-d4f6-4d93-8b4a-e2e1c7e96828 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:58.807872 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2aa57-8065-44c1-b6c8-f23c37f790e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:58.816616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2aa57-8065-44c1-b6c8-f23c37f790e3 service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:58.816616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2aa57-8065-44c1-b6c8-f23c37f790e3 service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:58.816616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2aa57-8065-44c1-b6c8-f23c37f790e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:58.821917 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-54f2aa57-8065-44c1-b6c8-f23c37f790e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:58.851616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2aa57-8065-44c1-b6c8-f23c37f790e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:58.864956 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2aa57-8065-44c1-b6c8-f23c37f790e3 service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:58.880005 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 669/1342] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:20:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:20:59.023987 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836a99a-a477-428d-8b5b-780b8703c184 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:59.025137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836a99a-a477-428d-8b5b-780b8703c184 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:59.025589 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836a99a-a477-428d-8b5b-780b8703c184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:59.034223 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17bb84-74bb-4fcd-a167-6bdc9869923c None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:59.034982 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17bb84-74bb-4fcd-a167-6bdc9869923c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:59.035474 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17bb84-74bb-4fcd-a167-6bdc9869923c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:59.085755 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-a836a99a-a477-428d-8b5b-780b8703c184 None None] MFA Rules not processed for user `e550c842cdbd4da396045c50a6d38866`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:59.099758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-fb17bb84-74bb-4fcd-a167-6bdc9869923c None None] MFA Rules not processed for user `c6e5113e504a4348b74d89a48ba50c17`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:59.103140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-fb17bb84-74bb-4fcd-a167-6bdc9869923c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:59.106389 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-a836a99a-a477-428d-8b5b-780b8703c184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:59.232024 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:20:59.248564 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 674/1343] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:59.309619 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 670/1344] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:59.321329 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3a456-beb6-4808-ab53-9fd37d99014c None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:59.321614 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3a456-beb6-4808-ab53-9fd37d99014c None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:59.322326 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3a456-beb6-4808-ab53-9fd37d99014c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:59.382876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-ffb3a456-beb6-4808-ab53-9fd37d99014c None None] MFA Rules not processed for user `c6e5113e504a4348b74d89a48ba50c17`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:20:59.398923 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ffb3a456-beb6-4808-ab53-9fd37d99014c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:59.426688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 675/1345] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:20:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:20:59.574196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3a456-beb6-4808-ab53-9fd37d99014c None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:59.574341 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17bb84-74bb-4fcd-a167-6bdc9869923c None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:20:59.575737 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ffb3a456-beb6-4808-ab53-9fd37d99014c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:20:59.798614 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3a456-beb6-4808-ab53-9fd37d99014c None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:59.817912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c621b0-c688-4fcb-968b-c57c49c573ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:59.818573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c621b0-c688-4fcb-968b-c57c49c573ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:59.818854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c621b0-c688-4fcb-968b-c57c49c573ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:59.819557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c621b0-c688-4fcb-968b-c57c49c573ae admin admin] PATH_INFO: `/v3/users/c25df0b554494b6085081a1e18a5a1d8` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:59.827770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c621b0-c688-4fcb-968b-c57c49c573ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=c25df0b554494b6085081a1e18a5a1d8)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:59.832760 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c621b0-c688-4fcb-968b-c57c49c573ae admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:59.847740 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17bb84-74bb-4fcd-a167-6bdc9869923c None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:20:59.878553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a91bd9-7f5f-4c91-bedc-dcfbf66aac79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:20:59.879201 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a91bd9-7f5f-4c91-bedc-dcfbf66aac79 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:20:59.879484 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a91bd9-7f5f-4c91-bedc-dcfbf66aac79 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:20:59.881756 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a91bd9-7f5f-4c91-bedc-dcfbf66aac79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:20:59.923956 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a91bd9-7f5f-4c91-bedc-dcfbf66aac79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:20:59.926180 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a91bd9-7f5f-4c91-bedc-dcfbf66aac79 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:20:59.976091 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 671/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:20:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:00.039780 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-f3c621b0-c688-4fcb-968b-c57c49c573ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c25df0b554494b6085081a1e18a5a1d8'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:00.080842 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a91bd9-7f5f-4c91-bedc-dcfbf66aac79 service nova] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:00.100461 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-f3c621b0-c688-4fcb-968b-c57c49c573ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c25df0b554494b6085081a1e18a5a1d8'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:00.106460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a91bd9-7f5f-4c91-bedc-dcfbf66aac79 service nova] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:00.132547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f9a56-33b4-4874-9af0-a07c22285ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:00.133525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f9a56-33b4-4874-9af0-a07c22285ca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:00.133893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f9a56-33b4-4874-9af0-a07c22285ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:00.134222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f9a56-33b4-4874-9af0-a07c22285ca1 admin admin] PATH_INFO: `/v3/users/88856db0aab648d9a9fe8cac873051e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:00.135469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f9a56-33b4-4874-9af0-a07c22285ca1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88856db0aab648d9a9fe8cac873051e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:00.153356 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f9a56-33b4-4874-9af0-a07c22285ca1 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:00.153575 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-f3c621b0-c688-4fcb-968b-c57c49c573ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c25df0b554494b6085081a1e18a5a1d8'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:00.157162 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-f3c621b0-c688-4fcb-968b-c57c49c573ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c25df0b554494b6085081a1e18a5a1d8'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:00.175192 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-f3c621b0-c688-4fcb-968b-c57c49c573ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c25df0b554494b6085081a1e18a5a1d8'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:00.180665 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:00.194008 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 676/1347] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:20:59 2022] DELETE /identity/v3/users/c25df0b554494b6085081a1e18a5a1d8 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:00.199348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eae8d2-b97f-4a7d-af08-aa3bdae0aa2b None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:00.199875 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eae8d2-b97f-4a7d-af08-aa3bdae0aa2b None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:00.201554 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eae8d2-b97f-4a7d-af08-aa3bdae0aa2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:00.247053 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-c6eae8d2-b97f-4a7d-af08-aa3bdae0aa2b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:00.249246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-c6eae8d2-b97f-4a7d-af08-aa3bdae0aa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:00.301150 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ae1f9a56-33b4-4874-9af0-a07c22285ca1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88856db0aab648d9a9fe8cac873051e9'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:00.303514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ae1f9a56-33b4-4874-9af0-a07c22285ca1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88856db0aab648d9a9fe8cac873051e9'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:00.337929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ae1f9a56-33b4-4874-9af0-a07c22285ca1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88856db0aab648d9a9fe8cac873051e9'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:00.363494 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ae1f9a56-33b4-4874-9af0-a07c22285ca1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88856db0aab648d9a9fe8cac873051e9'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:00.364560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ae1f9a56-33b4-4874-9af0-a07c22285ca1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88856db0aab648d9a9fe8cac873051e9'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:00.407376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 677/1348] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:00.412934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 672/1349] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:00 2022] DELETE /identity/v3/users/88856db0aab648d9a9fe8cac873051e9 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:00.417561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8274d3-ce11-4387-99cb-69be34982b92 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:00.417858 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8274d3-ce11-4387-99cb-69be34982b92 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:00.418141 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8274d3-ce11-4387-99cb-69be34982b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:00.435815 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f9a56-33b4-4874-9af0-a07c22285ca1 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:00.438057 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ae1f9a56-33b4-4874-9af0-a07c22285ca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:00.465531 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-bb8274d3-ce11-4387-99cb-69be34982b92 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:00.466793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-bb8274d3-ce11-4387-99cb-69be34982b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:00.554005 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:00.617343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 678/1350] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:00.629502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f9a56-33b4-4874-9af0-a07c22285ca1 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:00.646502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a78c1a-c659-495b-aca8-97c9f33b9139 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:00.648826 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a78c1a-c659-495b-aca8-97c9f33b9139 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:00.649344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a78c1a-c659-495b-aca8-97c9f33b9139 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:00.649832 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a78c1a-c659-495b-aca8-97c9f33b9139 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:00.664477 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8274d3-ce11-4387-99cb-69be34982b92 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:00.681683 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a78c1a-c659-495b-aca8-97c9f33b9139 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:00.683103 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8274d3-ce11-4387-99cb-69be34982b92 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:00.686451 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a78c1a-c659-495b-aca8-97c9f33b9139 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:00.695026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba0a3e-3d90-4953-b2f6-f5c6d0fa577f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:00.696868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba0a3e-3d90-4953-b2f6-f5c6d0fa577f service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:00.697861 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba0a3e-3d90-4953-b2f6-f5c6d0fa577f service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:00.698227 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba0a3e-3d90-4953-b2f6-f5c6d0fa577f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:00.725053 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 673/1351] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:21:00.736689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba0a3e-3d90-4953-b2f6-f5c6d0fa577f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:00.738409 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba0a3e-3d90-4953-b2f6-f5c6d0fa577f service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:00.763540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 679/1352] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:01.420235 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b71265-6ecd-4647-8854-39a55a3ece73 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:01.420527 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b71265-6ecd-4647-8854-39a55a3ece73 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:01.420803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b71265-6ecd-4647-8854-39a55a3ece73 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:01.423339 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 674/1353] 10.222.0.13 () {54 vars in 912 bytes} [Fri Oct 21 21:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:21:01.431375 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522f360-249b-4c1a-ad95-d59a978a8c99 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:01.432719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522f360-249b-4c1a-ad95-d59a978a8c99 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:01.432896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522f360-249b-4c1a-ad95-d59a978a8c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:01.490791 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-0522f360-249b-4c1a-ad95-d59a978a8c99 None None] MFA Rules not processed for user `d34dc4e7c5474b6b96977019328d8890`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:01.494386 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0522f360-249b-4c1a-ad95-d59a978a8c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:01.517778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 680/1354] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Oct 21 21:21:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:01.797143 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b71265-6ecd-4647-8854-39a55a3ece73 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:01.800617 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-60b71265-6ecd-4647-8854-39a55a3ece73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:01.835932 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b71265-6ecd-4647-8854-39a55a3ece73 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:01.839337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47594bf-e682-484c-97bb-967c8981fe80 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:01.839936 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47594bf-e682-484c-97bb-967c8981fe80 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:01.840437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47594bf-e682-484c-97bb-967c8981fe80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:01.857663 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4832858-dd02-4000-9bb0-2ad9e637bda3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:01.858677 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832858-dd02-4000-9bb0-2ad9e637bda3 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:01.864192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832858-dd02-4000-9bb0-2ad9e637bda3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:01.866501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4832858-dd02-4000-9bb0-2ad9e637bda3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:01.914810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4832858-dd02-4000-9bb0-2ad9e637bda3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:01.916607 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4832858-dd02-4000-9bb0-2ad9e637bda3 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:01.945091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-e47594bf-e682-484c-97bb-967c8981fe80 None None] MFA Rules not processed for user `64e9cccb41a04aef83ab417783d82fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:01.947827 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e47594bf-e682-484c-97bb-967c8981fe80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:01.950667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 675/1355] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:02.102924 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 681/1356] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:02.135618 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4832858-dd02-4000-9bb0-2ad9e637bda3 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:02.137303 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a4832858-dd02-4000-9bb0-2ad9e637bda3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:02.164286 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4832858-dd02-4000-9bb0-2ad9e637bda3 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:02.177803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad36ac3-8068-4a64-8d04-a94c05667945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:02.178689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad36ac3-8068-4a64-8d04-a94c05667945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:02.179110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad36ac3-8068-4a64-8d04-a94c05667945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:02.179684 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad36ac3-8068-4a64-8d04-a94c05667945 admin admin] PATH_INFO: `/v3/projects/3cf8471a7b094168b256a5cfe555fec8` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:02.188408 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad36ac3-8068-4a64-8d04-a94c05667945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cf8471a7b094168b256a5cfe555fec8)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:02.190476 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad36ac3-8068-4a64-8d04-a94c05667945 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:02.216425 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47594bf-e682-484c-97bb-967c8981fe80 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:02.217886 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e47594bf-e682-484c-97bb-967c8981fe80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:02.238917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47594bf-e682-484c-97bb-967c8981fe80 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:02.270387 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce047264-767c-4c3a-aefd-572fa6d98790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:02.271275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce047264-767c-4c3a-aefd-572fa6d98790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:02.271779 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce047264-767c-4c3a-aefd-572fa6d98790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:02.272461 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce047264-767c-4c3a-aefd-572fa6d98790 admin admin] PATH_INFO: `/v3/projects/57ba110632704f1f94a2621a0d2afbbe` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:02.275455 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce047264-767c-4c3a-aefd-572fa6d98790 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57ba110632704f1f94a2621a0d2afbbe)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:02.277854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce047264-767c-4c3a-aefd-572fa6d98790 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:02.284477 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:02.356924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5ad36ac3-8068-4a64-8d04-a94c05667945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cf8471a7b094168b256a5cfe555fec8', 'request_id': 'req-5ad36ac3-8068-4a64-8d04-a94c05667945'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:02.368745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5ad36ac3-8068-4a64-8d04-a94c05667945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cf8471a7b094168b256a5cfe555fec8', 'request_id': 'req-5ad36ac3-8068-4a64-8d04-a94c05667945'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:02.379827 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5ad36ac3-8068-4a64-8d04-a94c05667945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cf8471a7b094168b256a5cfe555fec8', 'request_id': 'req-5ad36ac3-8068-4a64-8d04-a94c05667945'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:02.402861 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5ad36ac3-8068-4a64-8d04-a94c05667945 admin admin] The token cache is being invalidate because project 3cf8471a7b094168b256a5cfe555fec8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:02.403398 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5ad36ac3-8068-4a64-8d04-a94c05667945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:02.404261 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5ad36ac3-8068-4a64-8d04-a94c05667945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:02.408369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 676/1357] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:02 2022] DELETE /identity/v3/projects/3cf8471a7b094168b256a5cfe555fec8 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:02.409564 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad36ac3-8068-4a64-8d04-a94c05667945 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:02.413943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-5ad36ac3-8068-4a64-8d04-a94c05667945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:02.420325 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ce047264-767c-4c3a-aefd-572fa6d98790 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57ba110632704f1f94a2621a0d2afbbe', 'request_id': 'req-ce047264-767c-4c3a-aefd-572fa6d98790'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:02.442086 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ce047264-767c-4c3a-aefd-572fa6d98790 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57ba110632704f1f94a2621a0d2afbbe', 'request_id': 'req-ce047264-767c-4c3a-aefd-572fa6d98790'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:02.458003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ce047264-767c-4c3a-aefd-572fa6d98790 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57ba110632704f1f94a2621a0d2afbbe', 'request_id': 'req-ce047264-767c-4c3a-aefd-572fa6d98790'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:02.476895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ce047264-767c-4c3a-aefd-572fa6d98790 admin admin] The token cache is being invalidate because project 57ba110632704f1f94a2621a0d2afbbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:02.477442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ce047264-767c-4c3a-aefd-572fa6d98790 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:02.479335 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ce047264-767c-4c3a-aefd-572fa6d98790 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:02.487318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 682/1358] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:02 2022] DELETE /identity/v3/projects/57ba110632704f1f94a2621a0d2afbbe => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:02.488408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce047264-767c-4c3a-aefd-572fa6d98790 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:02.494193 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ce047264-767c-4c3a-aefd-572fa6d98790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:02.589981 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad36ac3-8068-4a64-8d04-a94c05667945 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:02.703442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce047264-767c-4c3a-aefd-572fa6d98790 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:02.710349 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd48af-38e7-4e15-a1eb-510cdd8ece4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:02.711006 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd48af-38e7-4e15-a1eb-510cdd8ece4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:02.711286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd48af-38e7-4e15-a1eb-510cdd8ece4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:02.711560 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd48af-38e7-4e15-a1eb-510cdd8ece4d admin admin] PATH_INFO: `/v3/users/8fd019cd5127453f9c7200a007b1d305` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:02.720392 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd48af-38e7-4e15-a1eb-510cdd8ece4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fd019cd5127453f9c7200a007b1d305)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:02.722457 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd48af-38e7-4e15-a1eb-510cdd8ece4d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:02.743293 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:02.833161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ffcd48af-38e7-4e15-a1eb-510cdd8ece4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fd019cd5127453f9c7200a007b1d305'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:02.833952 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa798df3-5b02-4735-ba99-a795678c99e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:02.834601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa798df3-5b02-4735-ba99-a795678c99e3 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:02.835292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa798df3-5b02-4735-ba99-a795678c99e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:02.835859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa798df3-5b02-4735-ba99-a795678c99e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:02.841379 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-fa798df3-5b02-4735-ba99-a795678c99e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:02.862211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ffcd48af-38e7-4e15-a1eb-510cdd8ece4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fd019cd5127453f9c7200a007b1d305'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:02.890176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ffcd48af-38e7-4e15-a1eb-510cdd8ece4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fd019cd5127453f9c7200a007b1d305'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:02.891065 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ffcd48af-38e7-4e15-a1eb-510cdd8ece4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fd019cd5127453f9c7200a007b1d305'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:02.913113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ffcd48af-38e7-4e15-a1eb-510cdd8ece4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fd019cd5127453f9c7200a007b1d305'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:02.926608 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 683/1359] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:02 2022] DELETE /identity/v3/users/8fd019cd5127453f9c7200a007b1d305 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:02.928141 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffeca73-b1ed-40a2-9221-80f320f26a7e None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:02.928555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffeca73-b1ed-40a2-9221-80f320f26a7e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:02.934729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffeca73-b1ed-40a2-9221-80f320f26a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:02.995372 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-2ffeca73-b1ed-40a2-9221-80f320f26a7e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:02.997185 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2ffeca73-b1ed-40a2-9221-80f320f26a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:02.998988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa798df3-5b02-4735-ba99-a795678c99e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:03.004590 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa798df3-5b02-4735-ba99-a795678c99e3 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:03.017577 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:03.021657 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 677/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:03.145577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc95335-403f-4f11-b9b1-dd7a50cbd41b None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:03.145788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc95335-403f-4f11-b9b1-dd7a50cbd41b None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:03.147818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc95335-403f-4f11-b9b1-dd7a50cbd41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:03.158121 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 684/1361] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:03.166318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2067d5f-415a-414c-996e-8447b83fadc0 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:03.166614 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2067d5f-415a-414c-996e-8447b83fadc0 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:03.166892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2067d5f-415a-414c-996e-8447b83fadc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:03.230598 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-e2067d5f-415a-414c-996e-8447b83fadc0 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:03.231874 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e2067d5f-415a-414c-996e-8447b83fadc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:03.265646 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 685/1362] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:03.267632 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2067d5f-415a-414c-996e-8447b83fadc0 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:03.278284 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-9bc95335-403f-4f11-b9b1-dd7a50cbd41b None None] MFA Rules not processed for user `2346003afe74477496045e9bc72173d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:03.279536 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9bc95335-403f-4f11-b9b1-dd7a50cbd41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:03.306308 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2067d5f-415a-414c-996e-8447b83fadc0 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:03.323261 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d24eae-b19f-437e-b5b6-392219422a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:03.323556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d24eae-b19f-437e-b5b6-392219422a3d admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:03.324066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d24eae-b19f-437e-b5b6-392219422a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:03.324350 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d24eae-b19f-437e-b5b6-392219422a3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:03.325174 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d24eae-b19f-437e-b5b6-392219422a3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:21:03.325487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d24eae-b19f-437e-b5b6-392219422a3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:03.327113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d24eae-b19f-437e-b5b6-392219422a3d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:03.352106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 686/1363] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:21:03.356521 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d24eae-b19f-437e-b5b6-392219422a3d admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:03.384327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d24eae-b19f-437e-b5b6-392219422a3d admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:03.409115 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6512a3-5e30-4f0b-94c0-cbeb7375160b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:03.409756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6512a3-5e30-4f0b-94c0-cbeb7375160b admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:03.410034 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6512a3-5e30-4f0b-94c0-cbeb7375160b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:03.410297 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6512a3-5e30-4f0b-94c0-cbeb7375160b admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:03.410950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6512a3-5e30-4f0b-94c0-cbeb7375160b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:03.412977 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6512a3-5e30-4f0b-94c0-cbeb7375160b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:03.472425 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 678/1364] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Oct 21 21:21:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 21 21:21:03.474201 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4622b-20f3-4817-8aa7-3ccd179500cd None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:03.476431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4622b-20f3-4817-8aa7-3ccd179500cd None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:03.476979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4622b-20f3-4817-8aa7-3ccd179500cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:03.480255 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 687/1365] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:03 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:03.485176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6512a3-5e30-4f0b-94c0-cbeb7375160b admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:03.589466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-43d4622b-20f3-4817-8aa7-3ccd179500cd None None] MFA Rules not processed for user `dc5388b90d0847739330a095f4d5f106`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:03.596035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-43d4622b-20f3-4817-8aa7-3ccd179500cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:03.695832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6512a3-5e30-4f0b-94c0-cbeb7375160b admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:03.703581 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f6589-1328-4506-a689-1a10dbcb8843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:03.705515 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f6589-1328-4506-a689-1a10dbcb8843 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:03.706094 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f6589-1328-4506-a689-1a10dbcb8843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:03.707904 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f6589-1328-4506-a689-1a10dbcb8843 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:03.708973 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f6589-1328-4506-a689-1a10dbcb8843 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:03.713713 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f6589-1328-4506-a689-1a10dbcb8843 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:03.745701 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:03.779217 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 688/1366] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:03 2022] POST /identity/v3/users => generated 284 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:03.779217 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f6589-1328-4506-a689-1a10dbcb8843 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:03.788953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-cd5f6589-1328-4506-a689-1a10dbcb8843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:03.830141 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 679/1367] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:03.832096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4622b-20f3-4817-8aa7-3ccd179500cd None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:03.884441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4622b-20f3-4817-8aa7-3ccd179500cd None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:03.906571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced4f83-1c10-4a19-a1b7-0b393c21139e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:03.907357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced4f83-1c10-4a19-a1b7-0b393c21139e admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:03.907669 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced4f83-1c10-4a19-a1b7-0b393c21139e admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:03.907947 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced4f83-1c10-4a19-a1b7-0b393c21139e admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:03.908597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced4f83-1c10-4a19-a1b7-0b393c21139e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:03.913650 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced4f83-1c10-4a19-a1b7-0b393c21139e admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:03.947444 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 680/1368] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:03.953116 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced4f83-1c10-4a19-a1b7-0b393c21139e admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:04.014023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced4f83-1c10-4a19-a1b7-0b393c21139e admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:04.043171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d406ad-ff6b-4f1e-95aa-b3be26ae8890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:04.044369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d406ad-ff6b-4f1e-95aa-b3be26ae8890 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:04.045115 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d406ad-ff6b-4f1e-95aa-b3be26ae8890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:04.046366 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d406ad-ff6b-4f1e-95aa-b3be26ae8890 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:04.047869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d406ad-ff6b-4f1e-95aa-b3be26ae8890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:04.049838 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d406ad-ff6b-4f1e-95aa-b3be26ae8890 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:04.054709 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:04.077797 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 681/1369] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:04.083840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d406ad-ff6b-4f1e-95aa-b3be26ae8890 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:04.088873 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-11d406ad-ff6b-4f1e-95aa-b3be26ae8890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:04.170074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f6589-1328-4506-a689-1a10dbcb8843 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:04.204787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ee7d7-b8ff-49fd-a713-e2846d66208a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ce720c8df014f9a8af062138aecd1b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:04.208961 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ee7d7-b8ff-49fd-a713-e2846d66208a service swift] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:04.211103 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ee7d7-b8ff-49fd-a713-e2846d66208a service swift] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:04.212113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ee7d7-b8ff-49fd-a713-e2846d66208a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:04.365219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d406ad-ff6b-4f1e-95aa-b3be26ae8890 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:04.369339 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ee7d7-b8ff-49fd-a713-e2846d66208a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:04.371114 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ee7d7-b8ff-49fd-a713-e2846d66208a service swift] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:04.372484 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef42a7-dc2e-4834-93c5-760f895962f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:04.373314 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef42a7-dc2e-4834-93c5-760f895962f7 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:04.373884 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef42a7-dc2e-4834-93c5-760f895962f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:04.374329 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef42a7-dc2e-4834-93c5-760f895962f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:04.382597 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 689/1370] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Oct 21 21:21:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 21:21:04.383742 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ee7d7-b8ff-49fd-a713-e2846d66208a service swift] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:04.445300 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ee7d7-b8ff-49fd-a713-e2846d66208a service swift] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:04.462981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1403557-98d8-40f6-a989-c9fa0f778daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:04.464493 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1403557-98d8-40f6-a989-c9fa0f778daf admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:04.465292 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1403557-98d8-40f6-a989-c9fa0f778daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:04.465760 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1403557-98d8-40f6-a989-c9fa0f778daf admin admin] PATH_INFO: `/v3/projects/4cc9ad8fcdb44df1a65edc9fb7f31e92/users/a9ad8465e75f457aaa525394af05b11f/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:04.471384 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ef42a7-dc2e-4834-93c5-760f895962f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:04.475808 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ef42a7-dc2e-4834-93c5-760f895962f7 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:04.509633 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1403557-98d8-40f6-a989-c9fa0f778daf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cc9ad8fcdb44df1a65edc9fb7f31e92, user_id=a9ad8465e75f457aaa525394af05b11f, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:04.512540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1403557-98d8-40f6-a989-c9fa0f778daf admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:04.518429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 682/1371] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:04.564522 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 690/1372] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:04 2022] PUT /identity/v3/projects/4cc9ad8fcdb44df1a65edc9fb7f31e92/users/a9ad8465e75f457aaa525394af05b11f/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:04.568560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac20018-9f5f-47a8-888f-06db874f755b None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:04.569371 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac20018-9f5f-47a8-888f-06db874f755b None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:04.569952 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac20018-9f5f-47a8-888f-06db874f755b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:04.656185 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-fac20018-9f5f-47a8-888f-06db874f755b None None] MFA Rules not processed for user `a9ad8465e75f457aaa525394af05b11f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:04.657439 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-fac20018-9f5f-47a8-888f-06db874f755b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:04.848574 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 683/1373] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:04.861333 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1403557-98d8-40f6-a989-c9fa0f778daf admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:04.865712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e1403557-98d8-40f6-a989-c9fa0f778daf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:05.021149 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1403557-98d8-40f6-a989-c9fa0f778daf admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:05.026269 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd04a5-0426-44ba-bbf1-52673d82066f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:05.026913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd04a5-0426-44ba-bbf1-52673d82066f service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:05.027197 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd04a5-0426-44ba-bbf1-52673d82066f service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:05.027473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd04a5-0426-44ba-bbf1-52673d82066f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:05.048474 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd04a5-0426-44ba-bbf1-52673d82066f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:05.052350 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd04a5-0426-44ba-bbf1-52673d82066f service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:05.084590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 691/1374] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:05.621399 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dde1fb-79be-4c17-9668-bb72b250d53a None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:05.621910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dde1fb-79be-4c17-9668-bb72b250d53a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:05.621994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dde1fb-79be-4c17-9668-bb72b250d53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:05.717546 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-e1dde1fb-79be-4c17-9668-bb72b250d53a None None] MFA Rules not processed for user `00374760bfa648e48dedae8aadb61c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:05.718889 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-e1dde1fb-79be-4c17-9668-bb72b250d53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:05.854062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:06.100781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd04a5-0426-44ba-bbf1-52673d82066f service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:06.102463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-49fd04a5-0426-44ba-bbf1-52673d82066f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:06.141601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 684/1375] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 521 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:06.161390 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dde1fb-79be-4c17-9668-bb72b250d53a None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:06.431290 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd04a5-0426-44ba-bbf1-52673d82066f service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:06.450392 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2ebec-9909-4077-b7ce-76a0603e5e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:06.453989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2ebec-9909-4077-b7ce-76a0603e5e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:06.454355 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2ebec-9909-4077-b7ce-76a0603e5e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:06.454613 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2ebec-9909-4077-b7ce-76a0603e5e0b admin admin] PATH_INFO: `/v3/projects/db65f62e17404a0abd6046bfbc9f8b05` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:06.462146 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2ebec-9909-4077-b7ce-76a0603e5e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=db65f62e17404a0abd6046bfbc9f8b05)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:06.464194 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2ebec-9909-4077-b7ce-76a0603e5e0b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:06.480534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:06.508940 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dde1fb-79be-4c17-9668-bb72b250d53a None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:06.544055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc60546-0904-4250-bc88-e8493a2be799 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:06.544605 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc60546-0904-4250-bc88-e8493a2be799 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:06.546494 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc60546-0904-4250-bc88-e8493a2be799 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:06.547276 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc60546-0904-4250-bc88-e8493a2be799 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:06.591359 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc60546-0904-4250-bc88-e8493a2be799 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:06.602260 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc60546-0904-4250-bc88-e8493a2be799 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:06.649677 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 685/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:06.664902 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-33e2ebec-9909-4077-b7ce-76a0603e5e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db65f62e17404a0abd6046bfbc9f8b05', 'request_id': 'req-33e2ebec-9909-4077-b7ce-76a0603e5e0b'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:06.672358 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-33e2ebec-9909-4077-b7ce-76a0603e5e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db65f62e17404a0abd6046bfbc9f8b05', 'request_id': 'req-33e2ebec-9909-4077-b7ce-76a0603e5e0b'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:06.684331 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-33e2ebec-9909-4077-b7ce-76a0603e5e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db65f62e17404a0abd6046bfbc9f8b05', 'request_id': 'req-33e2ebec-9909-4077-b7ce-76a0603e5e0b'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:06.741985 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-33e2ebec-9909-4077-b7ce-76a0603e5e0b admin admin] The token cache is being invalidate because project db65f62e17404a0abd6046bfbc9f8b05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:06.744975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-33e2ebec-9909-4077-b7ce-76a0603e5e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:06.745950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-33e2ebec-9909-4077-b7ce-76a0603e5e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:06.751205 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 692/1377] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:06 2022] DELETE /identity/v3/projects/db65f62e17404a0abd6046bfbc9f8b05 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:06.759121 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93441f8-4ec2-403b-a7e3-3c91f3ece734 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:06.759807 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93441f8-4ec2-403b-a7e3-3c91f3ece734 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:06.760841 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93441f8-4ec2-403b-a7e3-3c91f3ece734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:06.827929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-d93441f8-4ec2-403b-a7e3-3c91f3ece734 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:06.833081 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d93441f8-4ec2-403b-a7e3-3c91f3ece734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:06.994142 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 686/1378] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:07.000834 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbb1cc-b11c-44c4-b8b6-514c0740b1db None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:07.001408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbb1cc-b11c-44c4-b8b6-514c0740b1db None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:07.002056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbb1cc-b11c-44c4-b8b6-514c0740b1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:07.058842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-8dfbb1cc-b11c-44c4-b8b6-514c0740b1db None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:07.064142 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8dfbb1cc-b11c-44c4-b8b6-514c0740b1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:07.093680 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 693/1379] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:07.099707 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93441f8-4ec2-403b-a7e3-3c91f3ece734 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:07.125478 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93441f8-4ec2-403b-a7e3-3c91f3ece734 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:07.156684 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6473f8-04c4-4f32-b23f-9f7255714239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:07.159297 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6473f8-04c4-4f32-b23f-9f7255714239 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:07.161787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6473f8-04c4-4f32-b23f-9f7255714239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:07.162710 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6473f8-04c4-4f32-b23f-9f7255714239 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:07.163652 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6473f8-04c4-4f32-b23f-9f7255714239 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:21:07.165998 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6473f8-04c4-4f32-b23f-9f7255714239 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:07.174216 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6473f8-04c4-4f32-b23f-9f7255714239 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:07.236946 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 687/1380] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:21:07.254181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbb1cc-b11c-44c4-b8b6-514c0740b1db None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:07.267119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbb1cc-b11c-44c4-b8b6-514c0740b1db None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:07.280532 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e7588-bc9a-48eb-b19e-68affe65a8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:07.281470 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e7588-bc9a-48eb-b19e-68affe65a8aa admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:07.282038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e7588-bc9a-48eb-b19e-68affe65a8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:07.284446 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e7588-bc9a-48eb-b19e-68affe65a8aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:07.285788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e7588-bc9a-48eb-b19e-68affe65a8aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:07.291378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e7588-bc9a-48eb-b19e-68affe65a8aa admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:07.388926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6473f8-04c4-4f32-b23f-9f7255714239 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:07.405381 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6473f8-04c4-4f32-b23f-9f7255714239 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:07.406064 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 694/1381] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:07 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:07.510932 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:07.542132 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c639c5-0538-4e4c-a17a-5b2b51f60085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:07.543270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c639c5-0538-4e4c-a17a-5b2b51f60085 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:07.543990 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c639c5-0538-4e4c-a17a-5b2b51f60085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:07.544458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c639c5-0538-4e4c-a17a-5b2b51f60085 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:07.545491 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c639c5-0538-4e4c-a17a-5b2b51f60085 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:07.548768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c639c5-0538-4e4c-a17a-5b2b51f60085 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:07.610205 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 688/1382] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:07 2022] POST /identity/v3/users => generated 294 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:07.611773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e7588-bc9a-48eb-b19e-68affe65a8aa admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:07.629184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e7588-bc9a-48eb-b19e-68affe65a8aa admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:07.643680 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad47c03-a575-43aa-9365-249059bbaae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:07.646540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad47c03-a575-43aa-9365-249059bbaae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:07.647314 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad47c03-a575-43aa-9365-249059bbaae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:07.648066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad47c03-a575-43aa-9365-249059bbaae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:07.650568 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad47c03-a575-43aa-9365-249059bbaae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:07.656483 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad47c03-a575-43aa-9365-249059bbaae3 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:07.686915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 695/1383] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:21:07.689325 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c639c5-0538-4e4c-a17a-5b2b51f60085 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:07.707891 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c639c5-0538-4e4c-a17a-5b2b51f60085 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:07.724789 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25411190-43b4-406e-9f8f-19ea6c414ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:07.725931 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25411190-43b4-406e-9f8f-19ea6c414ddf admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:07.726431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25411190-43b4-406e-9f8f-19ea6c414ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:07.726901 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25411190-43b4-406e-9f8f-19ea6c414ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:07.731508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25411190-43b4-406e-9f8f-19ea6c414ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:07.736210 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25411190-43b4-406e-9f8f-19ea6c414ddf admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:07.776419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 689/1384] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:07.780768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad47c03-a575-43aa-9365-249059bbaae3 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:07.795996 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad47c03-a575-43aa-9365-249059bbaae3 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:07.813278 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de78d22-70bc-43bf-b175-223a9af48cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:07.821581 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de78d22-70bc-43bf-b175-223a9af48cda admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:07.823925 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de78d22-70bc-43bf-b175-223a9af48cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:07.828005 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de78d22-70bc-43bf-b175-223a9af48cda admin admin] PATH_INFO: `/v3/projects/6909ea12494d4da0aba3c0fe9e082d81/users/9bd751cc19714633822c614e6654758d/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:07.905256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de78d22-70bc-43bf-b175-223a9af48cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=6909ea12494d4da0aba3c0fe9e082d81, user_id=9bd751cc19714633822c614e6654758d, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:07.916110 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de78d22-70bc-43bf-b175-223a9af48cda admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:07.964865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 696/1385] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:07 2022] PUT /identity/v3/projects/6909ea12494d4da0aba3c0fe9e082d81/users/9bd751cc19714633822c614e6654758d/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:07.976427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aacded-52fe-4098-99a9-5d93c3515ed9 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:07.977115 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aacded-52fe-4098-99a9-5d93c3515ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:07.977805 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aacded-52fe-4098-99a9-5d93c3515ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:08.079977 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-51aacded-52fe-4098-99a9-5d93c3515ed9 None None] MFA Rules not processed for user `9bd751cc19714633822c614e6654758d`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:08.089953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-51aacded-52fe-4098-99a9-5d93c3515ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:08.116013 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b3689-b640-46cf-a2d9-28050aaf74d6 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:08.116013 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b3689-b640-46cf-a2d9-28050aaf74d6 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:08.116013 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b3689-b640-46cf-a2d9-28050aaf74d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:08.197434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-687b3689-b640-46cf-a2d9-28050aaf74d6 None None] MFA Rules not processed for user `d34dc4e7c5474b6b96977019328d8890`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:08.203405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-687b3689-b640-46cf-a2d9-28050aaf74d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:08.502443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 690/1386] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 528 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:08.508330 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aacded-52fe-4098-99a9-5d93c3515ed9 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:08.515660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 697/1387] 10.222.0.44 () {58 vars in 1015 bytes} [Fri Oct 21 21:21:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:08.605874 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b3689-b640-46cf-a2d9-28050aaf74d6 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:08.614462 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-687b3689-b640-46cf-a2d9-28050aaf74d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:08.660021 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b3689-b640-46cf-a2d9-28050aaf74d6 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:08.682926 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64359b32-fd76-492b-b5dc-b02d809817cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:08.685065 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64359b32-fd76-492b-b5dc-b02d809817cd service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:08.686085 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64359b32-fd76-492b-b5dc-b02d809817cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:08.689420 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64359b32-fd76-492b-b5dc-b02d809817cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:08.712418 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64359b32-fd76-492b-b5dc-b02d809817cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:08.714247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64359b32-fd76-492b-b5dc-b02d809817cd service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:08.732729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 698/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:21:08.887796 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aacded-52fe-4098-99a9-5d93c3515ed9 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:08.912335 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369fb44-7383-4bf3-9204-2edf8a4f8bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:08.912952 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369fb44-7383-4bf3-9204-2edf8a4f8bce admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:08.913343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369fb44-7383-4bf3-9204-2edf8a4f8bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:08.921336 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369fb44-7383-4bf3-9204-2edf8a4f8bce admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:08.935054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369fb44-7383-4bf3-9204-2edf8a4f8bce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:08.935054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369fb44-7383-4bf3-9204-2edf8a4f8bce admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:09.053403 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 691/1389] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:08 2022] POST /identity/v3/projects => generated 389 bytes in 543 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:09.054844 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64359b32-fd76-492b-b5dc-b02d809817cd service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:09.082473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64359b32-fd76-492b-b5dc-b02d809817cd service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:09.184085 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:09.412128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ced16-ed6e-47d5-aa2d-43b644f82a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:09.413307 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ced16-ed6e-47d5-aa2d-43b644f82a8d admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:09.413781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ced16-ed6e-47d5-aa2d-43b644f82a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:09.414065 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ced16-ed6e-47d5-aa2d-43b644f82a8d admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:09.415466 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ced16-ed6e-47d5-aa2d-43b644f82a8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:09.418550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ced16-ed6e-47d5-aa2d-43b644f82a8d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:09.499836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 699/1390] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:09 2022] POST /identity/v3/users => generated 301 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:09.516175 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369fb44-7383-4bf3-9204-2edf8a4f8bce admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:09.536081 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369fb44-7383-4bf3-9204-2edf8a4f8bce admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:09.548529 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388bd02-d47f-4707-9183-9d2842622dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:09.550254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388bd02-d47f-4707-9183-9d2842622dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:09.551597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388bd02-d47f-4707-9183-9d2842622dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:09.552083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388bd02-d47f-4707-9183-9d2842622dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:09.556369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8388bd02-d47f-4707-9183-9d2842622dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:09.559773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8388bd02-d47f-4707-9183-9d2842622dd1 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:09.601997 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 692/1391] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:09.603203 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ced16-ed6e-47d5-aa2d-43b644f82a8d admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:09.615639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ced16-ed6e-47d5-aa2d-43b644f82a8d admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:09.639499 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf420e53-d545-4b0b-b199-f53dff6b7be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:09.651618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf420e53-d545-4b0b-b199-f53dff6b7be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:09.651618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf420e53-d545-4b0b-b199-f53dff6b7be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:09.651618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf420e53-d545-4b0b-b199-f53dff6b7be0 admin admin] PATH_INFO: `/v3/projects/b3270bc4a6be4ce09c7c6595c4913b57/users/9ada01542a0f46f287fafd72252aae45/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:09.710117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf420e53-d545-4b0b-b199-f53dff6b7be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3270bc4a6be4ce09c7c6595c4913b57, user_id=9ada01542a0f46f287fafd72252aae45, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:09.720199 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf420e53-d545-4b0b-b199-f53dff6b7be0 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:09.781790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 700/1392] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:09 2022] PUT /identity/v3/projects/b3270bc4a6be4ce09c7c6595c4913b57/users/9ada01542a0f46f287fafd72252aae45/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:09.783673 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388bd02-d47f-4707-9183-9d2842622dd1 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:09.804967 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388bd02-d47f-4707-9183-9d2842622dd1 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:09.852737 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:09.985790 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78dfaf-791a-4e63-b63a-1dba2d644a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:09.986447 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78dfaf-791a-4e63-b63a-1dba2d644a87 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:09.986704 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78dfaf-791a-4e63-b63a-1dba2d644a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:09.986980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78dfaf-791a-4e63-b63a-1dba2d644a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:09.990899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78dfaf-791a-4e63-b63a-1dba2d644a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:09.997738 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78dfaf-791a-4e63-b63a-1dba2d644a87 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:10.030992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 693/1393] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:10.033302 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf420e53-d545-4b0b-b199-f53dff6b7be0 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:10.049230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf420e53-d545-4b0b-b199-f53dff6b7be0 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:10.064569 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936706f-57e6-4fc6-b523-037e37e57612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:10.067411 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936706f-57e6-4fc6-b523-037e37e57612 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:10.068064 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936706f-57e6-4fc6-b523-037e37e57612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:10.068351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936706f-57e6-4fc6-b523-037e37e57612 admin admin] PATH_INFO: `/v3/projects/b3270bc4a6be4ce09c7c6595c4913b57/users/9ada01542a0f46f287fafd72252aae45/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:10.075581 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936706f-57e6-4fc6-b523-037e37e57612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3270bc4a6be4ce09c7c6595c4913b57, user_id=9ada01542a0f46f287fafd72252aae45, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:10.083134 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936706f-57e6-4fc6-b523-037e37e57612 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:10.125431 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 701/1394] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:10 2022] PUT /identity/v3/projects/b3270bc4a6be4ce09c7c6595c4913b57/users/9ada01542a0f46f287fafd72252aae45/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:10.141617 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da98a8-8160-464b-ade6-f614978de681 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:10.141919 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da98a8-8160-464b-ade6-f614978de681 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:10.142200 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da98a8-8160-464b-ade6-f614978de681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:10.208717 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-62da98a8-8160-464b-ade6-f614978de681 None None] MFA Rules not processed for user `9ada01542a0f46f287fafd72252aae45`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:10.210613 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-62da98a8-8160-464b-ade6-f614978de681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:10.349093 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 694/1395] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:10 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:10.361139 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936706f-57e6-4fc6-b523-037e37e57612 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:10.368157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0936706f-57e6-4fc6-b523-037e37e57612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:10.681622 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936706f-57e6-4fc6-b523-037e37e57612 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:10.727750 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b443db7-9798-46e7-af8e-559e6d3bba20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:10.727750 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b443db7-9798-46e7-af8e-559e6d3bba20 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:10.727750 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b443db7-9798-46e7-af8e-559e6d3bba20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:10.727750 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b443db7-9798-46e7-af8e-559e6d3bba20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:10.783802 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b443db7-9798-46e7-af8e-559e6d3bba20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:10.785729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b443db7-9798-46e7-af8e-559e6d3bba20 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:10.844509 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 702/1396] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:10 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:12.080474 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da98a8-8160-464b-ade6-f614978de681 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:12.339304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da98a8-8160-464b-ade6-f614978de681 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:12.359705 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a49b9-bb89-4fe3-89d7-12ac3c7d3604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:12.360375 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a49b9-bb89-4fe3-89d7-12ac3c7d3604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:12.360661 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a49b9-bb89-4fe3-89d7-12ac3c7d3604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:12.360986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a49b9-bb89-4fe3-89d7-12ac3c7d3604 admin admin] PATH_INFO: `/v3/users/9bd751cc19714633822c614e6654758d` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:12.362217 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a49b9-bb89-4fe3-89d7-12ac3c7d3604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bd751cc19714633822c614e6654758d)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:12.364940 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a49b9-bb89-4fe3-89d7-12ac3c7d3604 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:12.530324 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:12.538098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-fa4a49b9-bb89-4fe3-89d7-12ac3c7d3604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bd751cc19714633822c614e6654758d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:12.541171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-fa4a49b9-bb89-4fe3-89d7-12ac3c7d3604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bd751cc19714633822c614e6654758d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:12.582856 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-fa4a49b9-bb89-4fe3-89d7-12ac3c7d3604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bd751cc19714633822c614e6654758d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:12.604675 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-fa4a49b9-bb89-4fe3-89d7-12ac3c7d3604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bd751cc19714633822c614e6654758d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:12.605495 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-fa4a49b9-bb89-4fe3-89d7-12ac3c7d3604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bd751cc19714633822c614e6654758d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:12.642457 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 695/1397] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:12 2022] DELETE /identity/v3/users/9bd751cc19714633822c614e6654758d => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:12.643666 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6dc22-c201-453e-bf09-43c74a6e1811 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:12.644605 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6dc22-c201-453e-bf09-43c74a6e1811 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:12.644817 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6dc22-c201-453e-bf09-43c74a6e1811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:12.729410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-43b6dc22-c201-453e-bf09-43c74a6e1811 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:12.730588 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-43b6dc22-c201-453e-bf09-43c74a6e1811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:12.941794 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 703/1398] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:12.981105 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a49b9-bb89-4fe3-89d7-12ac3c7d3604 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:12.982600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-fa4a49b9-bb89-4fe3-89d7-12ac3c7d3604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:13.205048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a49b9-bb89-4fe3-89d7-12ac3c7d3604 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:13.221502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922211ee-95d2-4ca9-bf63-e4909a858948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:13.223781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922211ee-95d2-4ca9-bf63-e4909a858948 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:13.225621 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922211ee-95d2-4ca9-bf63-e4909a858948 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:13.226772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922211ee-95d2-4ca9-bf63-e4909a858948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:13.280484 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922211ee-95d2-4ca9-bf63-e4909a858948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:13.286093 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922211ee-95d2-4ca9-bf63-e4909a858948 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:13.328303 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 696/1399] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:14.743695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6dc22-c201-453e-bf09-43c74a6e1811 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:14.746663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-43b6dc22-c201-453e-bf09-43c74a6e1811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:14.782465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6dc22-c201-453e-bf09-43c74a6e1811 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:14.798655 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6fa9c-54cf-4dbb-9f7e-5e2f3e4fb911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:14.800850 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6fa9c-54cf-4dbb-9f7e-5e2f3e4fb911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:14.801427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6fa9c-54cf-4dbb-9f7e-5e2f3e4fb911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:14.802004 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6fa9c-54cf-4dbb-9f7e-5e2f3e4fb911 admin admin] PATH_INFO: `/v3/users/3f22e1859b1c4952a0f77cb5f7e2e586` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:14.803484 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6fa9c-54cf-4dbb-9f7e-5e2f3e4fb911 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f22e1859b1c4952a0f77cb5f7e2e586)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:14.806430 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6fa9c-54cf-4dbb-9f7e-5e2f3e4fb911 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:14.845307 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:14.925899 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-99f6fa9c-54cf-4dbb-9f7e-5e2f3e4fb911 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f22e1859b1c4952a0f77cb5f7e2e586'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:14.966079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-99f6fa9c-54cf-4dbb-9f7e-5e2f3e4fb911 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f22e1859b1c4952a0f77cb5f7e2e586'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:14.987311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-99f6fa9c-54cf-4dbb-9f7e-5e2f3e4fb911 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f22e1859b1c4952a0f77cb5f7e2e586'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:14.988022 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-99f6fa9c-54cf-4dbb-9f7e-5e2f3e4fb911 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f22e1859b1c4952a0f77cb5f7e2e586'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:15.015247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-99f6fa9c-54cf-4dbb-9f7e-5e2f3e4fb911 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f22e1859b1c4952a0f77cb5f7e2e586'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:15.039033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 704/1400] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:14 2022] DELETE /identity/v3/users/3f22e1859b1c4952a0f77cb5f7e2e586 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:15.043969 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cff9ba-afa4-4523-93e8-e467b9c5aa68 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:15.044510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cff9ba-afa4-4523-93e8-e467b9c5aa68 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:15.044510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cff9ba-afa4-4523-93e8-e467b9c5aa68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:15.092009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-06cff9ba-afa4-4523-93e8-e467b9c5aa68 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:15.093325 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-06cff9ba-afa4-4523-93e8-e467b9c5aa68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:15.229232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 697/1401] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:15.319069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6fa9c-54cf-4dbb-9f7e-5e2f3e4fb911 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:15.320647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-99f6fa9c-54cf-4dbb-9f7e-5e2f3e4fb911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:15.478501 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6fa9c-54cf-4dbb-9f7e-5e2f3e4fb911 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:15.488781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6e0af-cc75-4ae0-bab3-c47a1ad30cf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:15.489430 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6e0af-cc75-4ae0-bab3-c47a1ad30cf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:15.489711 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6e0af-cc75-4ae0-bab3-c47a1ad30cf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:15.489988 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6e0af-cc75-4ae0-bab3-c47a1ad30cf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:15.517579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6e0af-cc75-4ae0-bab3-c47a1ad30cf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:15.521314 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6e0af-cc75-4ae0-bab3-c47a1ad30cf5 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:15.555491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 705/1402] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:17.623476 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219859c6-4ca2-404e-9b09-dcda47ccdf66 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:17.624039 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219859c6-4ca2-404e-9b09-dcda47ccdf66 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:17.624164 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219859c6-4ca2-404e-9b09-dcda47ccdf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:17.664960 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-219859c6-4ca2-404e-9b09-dcda47ccdf66 None None] MFA Rules not processed for user `64e9cccb41a04aef83ab417783d82fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:17.675865 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-219859c6-4ca2-404e-9b09-dcda47ccdf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:17.862378 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 698/1403] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:17.880498 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6e0af-cc75-4ae0-bab3-c47a1ad30cf5 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:17.883435 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-57b6e0af-cc75-4ae0-bab3-c47a1ad30cf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:17.927834 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6e0af-cc75-4ae0-bab3-c47a1ad30cf5 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:17.966297 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc1c7c-7a85-41dd-976e-384f23f9e158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:17.971475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc1c7c-7a85-41dd-976e-384f23f9e158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:17.976137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc1c7c-7a85-41dd-976e-384f23f9e158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:17.976436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc1c7c-7a85-41dd-976e-384f23f9e158 admin admin] PATH_INFO: `/v3/projects/6909ea12494d4da0aba3c0fe9e082d81` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:17.984981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc1c7c-7a85-41dd-976e-384f23f9e158 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6909ea12494d4da0aba3c0fe9e082d81)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:17.991333 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc1c7c-7a85-41dd-976e-384f23f9e158 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:18.124264 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-45fc1c7c-7a85-41dd-976e-384f23f9e158 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6909ea12494d4da0aba3c0fe9e082d81', 'request_id': 'req-45fc1c7c-7a85-41dd-976e-384f23f9e158'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:18.157792 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-45fc1c7c-7a85-41dd-976e-384f23f9e158 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6909ea12494d4da0aba3c0fe9e082d81', 'request_id': 'req-45fc1c7c-7a85-41dd-976e-384f23f9e158'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:18.190132 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-45fc1c7c-7a85-41dd-976e-384f23f9e158 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6909ea12494d4da0aba3c0fe9e082d81', 'request_id': 'req-45fc1c7c-7a85-41dd-976e-384f23f9e158'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:18.192764 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fb819-5c76-4270-88e1-4657b1d7f848 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:18.193206 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fb819-5c76-4270-88e1-4657b1d7f848 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:18.193650 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fb819-5c76-4270-88e1-4657b1d7f848 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:18.195471 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 699/1404] 10.222.0.13 () {54 vars in 921 bytes} [Fri Oct 21 21:21:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:21:18.239314 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a3ca6-273d-4133-8c27-182efee144fb None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:18.239655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a3ca6-273d-4133-8c27-182efee144fb None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:18.239909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a3ca6-273d-4133-8c27-182efee144fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:18.245933 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-45fc1c7c-7a85-41dd-976e-384f23f9e158 admin admin] The token cache is being invalidate because project 6909ea12494d4da0aba3c0fe9e082d81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:18.247320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-45fc1c7c-7a85-41dd-976e-384f23f9e158 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:18.250874 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-45fc1c7c-7a85-41dd-976e-384f23f9e158 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:18.255360 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 706/1405] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:17 2022] DELETE /identity/v3/projects/6909ea12494d4da0aba3c0fe9e082d81 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:18.260347 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc1c7c-7a85-41dd-976e-384f23f9e158 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:18.265048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-45fc1c7c-7a85-41dd-976e-384f23f9e158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:18.276607 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-669a3ca6-273d-4133-8c27-182efee144fb None None] MFA Rules not processed for user `12dad9f016a740b98538722e1b723823`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:18.277844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-669a3ca6-273d-4133-8c27-182efee144fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:18.296267 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:18.317530 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:18.436755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 700/1406] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Oct 21 21:21:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:18.447905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc1c7c-7a85-41dd-976e-384f23f9e158 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:18.462917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b61ff1-4758-4665-8fcf-0fe9c63d71c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:18.463902 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b61ff1-4758-4665-8fcf-0fe9c63d71c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:18.464432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b61ff1-4758-4665-8fcf-0fe9c63d71c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:18.464779 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b61ff1-4758-4665-8fcf-0fe9c63d71c0 admin admin] PATH_INFO: `/v3/users/9ada01542a0f46f287fafd72252aae45` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:18.465805 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411c7d0-3713-4145-96b2-b86e66b87222 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:18.466091 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411c7d0-3713-4145-96b2-b86e66b87222 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:18.466366 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411c7d0-3713-4145-96b2-b86e66b87222 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:18.468196 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 701/1407] 10.222.0.13 () {54 vars in 921 bytes} [Fri Oct 21 21:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:21:18.470395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b61ff1-4758-4665-8fcf-0fe9c63d71c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ada01542a0f46f287fafd72252aae45)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:18.472744 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b61ff1-4758-4665-8fcf-0fe9c63d71c0 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:18.589189 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-42b61ff1-4758-4665-8fcf-0fe9c63d71c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ada01542a0f46f287fafd72252aae45'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:18.620934 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-42b61ff1-4758-4665-8fcf-0fe9c63d71c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ada01542a0f46f287fafd72252aae45'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:18.638358 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-42b61ff1-4758-4665-8fcf-0fe9c63d71c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ada01542a0f46f287fafd72252aae45'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:18.639118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-42b61ff1-4758-4665-8fcf-0fe9c63d71c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ada01542a0f46f287fafd72252aae45'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:18.657570 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-42b61ff1-4758-4665-8fcf-0fe9c63d71c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ada01542a0f46f287fafd72252aae45'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:18.665840 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 707/1408] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:18 2022] DELETE /identity/v3/users/9ada01542a0f46f287fafd72252aae45 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:18.964343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e992ff-5e49-4eae-b14d-9713cf6ceebc None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:18.964908 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e992ff-5e49-4eae-b14d-9713cf6ceebc None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:18.964908 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e992ff-5e49-4eae-b14d-9713cf6ceebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:18.995423 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b61ff1-4758-4665-8fcf-0fe9c63d71c0 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:18.997326 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-42b61ff1-4758-4665-8fcf-0fe9c63d71c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:19.000488 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-d2e992ff-5e49-4eae-b14d-9713cf6ceebc None None] MFA Rules not processed for user `12dad9f016a740b98538722e1b723823`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:19.002070 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d2e992ff-5e49-4eae-b14d-9713cf6ceebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:19.167257 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b61ff1-4758-4665-8fcf-0fe9c63d71c0 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:19.167446 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 702/1409] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Oct 21 21:21:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:19.188522 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bca61-51f8-49fc-94d4-7f7b7443f358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:19.189383 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bca61-51f8-49fc-94d4-7f7b7443f358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:19.189763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bca61-51f8-49fc-94d4-7f7b7443f358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:19.190137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bca61-51f8-49fc-94d4-7f7b7443f358 admin admin] PATH_INFO: `/v3/projects/4100f18f980e45618e20917e6323eb2f` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:19.191506 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bca61-51f8-49fc-94d4-7f7b7443f358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4100f18f980e45618e20917e6323eb2f)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:19.194116 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bca61-51f8-49fc-94d4-7f7b7443f358 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:19.296627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:19.326416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-237bca61-51f8-49fc-94d4-7f7b7443f358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4100f18f980e45618e20917e6323eb2f', 'request_id': 'req-237bca61-51f8-49fc-94d4-7f7b7443f358'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:19.346771 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-237bca61-51f8-49fc-94d4-7f7b7443f358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4100f18f980e45618e20917e6323eb2f', 'request_id': 'req-237bca61-51f8-49fc-94d4-7f7b7443f358'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:19.366661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-237bca61-51f8-49fc-94d4-7f7b7443f358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4100f18f980e45618e20917e6323eb2f', 'request_id': 'req-237bca61-51f8-49fc-94d4-7f7b7443f358'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:19.381956 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e992ff-5e49-4eae-b14d-9713cf6ceebc None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:19.383389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d2e992ff-5e49-4eae-b14d-9713cf6ceebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:19.406644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e992ff-5e49-4eae-b14d-9713cf6ceebc None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:19.414585 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-237bca61-51f8-49fc-94d4-7f7b7443f358 admin admin] The token cache is being invalidate because project 4100f18f980e45618e20917e6323eb2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:19.415098 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-237bca61-51f8-49fc-94d4-7f7b7443f358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:19.416098 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-237bca61-51f8-49fc-94d4-7f7b7443f358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:19.432944 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 708/1410] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:18 2022] DELETE /identity/v3/projects/4100f18f980e45618e20917e6323eb2f => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:19.437239 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bca61-51f8-49fc-94d4-7f7b7443f358 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:19.440058 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-237bca61-51f8-49fc-94d4-7f7b7443f358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:19.601256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddb9e6-ecea-4da4-997a-94920e20f16a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:19.602008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddb9e6-ecea-4da4-997a-94920e20f16a service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:19.602291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddb9e6-ecea-4da4-997a-94920e20f16a service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:19.602567 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddb9e6-ecea-4da4-997a-94920e20f16a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:19.612539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-61ddb9e6-ecea-4da4-997a-94920e20f16a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:19.637956 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ddb9e6-ecea-4da4-997a-94920e20f16a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:19.639620 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ddb9e6-ecea-4da4-997a-94920e20f16a service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:19.657392 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 703/1411] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:21:19.691282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bca61-51f8-49fc-94d4-7f7b7443f358 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:19.705076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d686874-4375-47e1-8554-2d078717fd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:19.708077 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d686874-4375-47e1-8554-2d078717fd47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:19.709230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d686874-4375-47e1-8554-2d078717fd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:19.709572 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d686874-4375-47e1-8554-2d078717fd47 admin admin] PATH_INFO: `/v3/users/51775c01d12b487d9015dcb6fd7265a8` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:19.712252 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d686874-4375-47e1-8554-2d078717fd47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51775c01d12b487d9015dcb6fd7265a8)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:19.717854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d686874-4375-47e1-8554-2d078717fd47 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:19.794782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500427a5-853d-45ab-8104-5400d91966f2 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:19.795168 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500427a5-853d-45ab-8104-5400d91966f2 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:19.795402 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500427a5-853d-45ab-8104-5400d91966f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:19.836691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-500427a5-853d-45ab-8104-5400d91966f2 None None] MFA Rules not processed for user `dc5388b90d0847739330a095f4d5f106`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:19.839933 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-500427a5-853d-45ab-8104-5400d91966f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:19.854306 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2d686874-4375-47e1-8554-2d078717fd47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51775c01d12b487d9015dcb6fd7265a8'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:19.871343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:19.895319 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2d686874-4375-47e1-8554-2d078717fd47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51775c01d12b487d9015dcb6fd7265a8'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:19.910451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2d686874-4375-47e1-8554-2d078717fd47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51775c01d12b487d9015dcb6fd7265a8'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:19.911418 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2d686874-4375-47e1-8554-2d078717fd47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51775c01d12b487d9015dcb6fd7265a8'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:19.930010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 704/1412] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:19.935397 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2d686874-4375-47e1-8554-2d078717fd47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51775c01d12b487d9015dcb6fd7265a8'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:19.943885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 709/1413] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:19 2022] DELETE /identity/v3/users/51775c01d12b487d9015dcb6fd7265a8 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:20.403939 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500427a5-853d-45ab-8104-5400d91966f2 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:20.406023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-500427a5-853d-45ab-8104-5400d91966f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:20.429412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500427a5-853d-45ab-8104-5400d91966f2 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:20.475686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d686874-4375-47e1-8554-2d078717fd47 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:20.477448 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2d686874-4375-47e1-8554-2d078717fd47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:20.599425 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af52121-7d20-45ae-9f85-162707388b16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:20.600305 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af52121-7d20-45ae-9f85-162707388b16 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:20.600531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af52121-7d20-45ae-9f85-162707388b16 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:20.600803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af52121-7d20-45ae-9f85-162707388b16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:20.613280 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:20.613422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-6af52121-7d20-45ae-9f85-162707388b16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:20.647755 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d686874-4375-47e1-8554-2d078717fd47 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:20.647898 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af52121-7d20-45ae-9f85-162707388b16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:20.650297 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af52121-7d20-45ae-9f85-162707388b16 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:20.653047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe362fb5-a154-49d9-bb3d-989897d5f8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:20.653873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe362fb5-a154-49d9-bb3d-989897d5f8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:20.654308 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe362fb5-a154-49d9-bb3d-989897d5f8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:20.654735 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe362fb5-a154-49d9-bb3d-989897d5f8b5 admin admin] PATH_INFO: `/v3/projects/b3270bc4a6be4ce09c7c6595c4913b57` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:20.656766 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe362fb5-a154-49d9-bb3d-989897d5f8b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3270bc4a6be4ce09c7c6595c4913b57)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:20.660476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe362fb5-a154-49d9-bb3d-989897d5f8b5 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:20.684090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 705/1414] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:20.685828 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72905793-9e19-49fd-a93c-9175a714c6fd None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:20.686126 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72905793-9e19-49fd-a93c-9175a714c6fd None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:20.686445 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72905793-9e19-49fd-a93c-9175a714c6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:20.710985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-72905793-9e19-49fd-a93c-9175a714c6fd None None] MFA Rules not processed for user `00374760bfa648e48dedae8aadb61c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:20.712391 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-72905793-9e19-49fd-a93c-9175a714c6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:20.780446 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-fe362fb5-a154-49d9-bb3d-989897d5f8b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3270bc4a6be4ce09c7c6595c4913b57', 'request_id': 'req-fe362fb5-a154-49d9-bb3d-989897d5f8b5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:20.789259 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-fe362fb5-a154-49d9-bb3d-989897d5f8b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3270bc4a6be4ce09c7c6595c4913b57', 'request_id': 'req-fe362fb5-a154-49d9-bb3d-989897d5f8b5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:20.803234 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-fe362fb5-a154-49d9-bb3d-989897d5f8b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3270bc4a6be4ce09c7c6595c4913b57', 'request_id': 'req-fe362fb5-a154-49d9-bb3d-989897d5f8b5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:20.808489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 706/1415] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:20.809781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72905793-9e19-49fd-a93c-9175a714c6fd None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:20.814114 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-72905793-9e19-49fd-a93c-9175a714c6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:20.829501 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-fe362fb5-a154-49d9-bb3d-989897d5f8b5 admin admin] The token cache is being invalidate because project b3270bc4a6be4ce09c7c6595c4913b57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:20.830094 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-fe362fb5-a154-49d9-bb3d-989897d5f8b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:20.831135 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-fe362fb5-a154-49d9-bb3d-989897d5f8b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:20.835439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 710/1416] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:20 2022] DELETE /identity/v3/projects/b3270bc4a6be4ce09c7c6595c4913b57 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:20.840509 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684de17a-16c5-47b6-8dd8-363f1743aa16 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:20.841084 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684de17a-16c5-47b6-8dd8-363f1743aa16 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:20.841577 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684de17a-16c5-47b6-8dd8-363f1743aa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:20.857385 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72905793-9e19-49fd-a93c-9175a714c6fd None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:20.869321 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-684de17a-16c5-47b6-8dd8-363f1743aa16 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:20.870569 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-684de17a-16c5-47b6-8dd8-363f1743aa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:20.990537 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:21.007162 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 711/1417] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:21.016009 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f5d6f-0a95-488d-9c7f-8ca7a86a57fa None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:21.016703 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f5d6f-0a95-488d-9c7f-8ca7a86a57fa None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:21.017787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f5d6f-0a95-488d-9c7f-8ca7a86a57fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:21.021216 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee88d8-f473-4b93-9af0-74cc982f218e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:21.021881 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee88d8-f473-4b93-9af0-74cc982f218e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:21.022169 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee88d8-f473-4b93-9af0-74cc982f218e admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:21.023344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee88d8-f473-4b93-9af0-74cc982f218e admin admin] PATH_INFO: `/v3/users/a9ad8465e75f457aaa525394af05b11f` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:21.025808 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee88d8-f473-4b93-9af0-74cc982f218e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9ad8465e75f457aaa525394af05b11f)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:21.028828 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee88d8-f473-4b93-9af0-74cc982f218e admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:21.053966 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-697f5d6f-0a95-488d-9c7f-8ca7a86a57fa None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:21.056178 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-697f5d6f-0a95-488d-9c7f-8ca7a86a57fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:21.068293 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 712/1418] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:21.072295 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f5d6f-0a95-488d-9c7f-8ca7a86a57fa None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:21.093937 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f5d6f-0a95-488d-9c7f-8ca7a86a57fa None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:21.111949 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bf148-4803-45f4-a3ea-2b49fd16da72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:21.113269 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bf148-4803-45f4-a3ea-2b49fd16da72 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:21.113487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bf148-4803-45f4-a3ea-2b49fd16da72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:21.113793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bf148-4803-45f4-a3ea-2b49fd16da72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:21.114444 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bf148-4803-45f4-a3ea-2b49fd16da72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:21:21.114831 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bf148-4803-45f4-a3ea-2b49fd16da72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:21.116542 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bf148-4803-45f4-a3ea-2b49fd16da72 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:21.130962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3dee88d8-f473-4b93-9af0-74cc982f218e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9ad8465e75f457aaa525394af05b11f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:21.132431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3dee88d8-f473-4b93-9af0-74cc982f218e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9ad8465e75f457aaa525394af05b11f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:21.137965 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 713/1419] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:21:21.139250 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bf148-4803-45f4-a3ea-2b49fd16da72 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:21.143218 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-cc0bf148-4803-45f4-a3ea-2b49fd16da72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:21.165443 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bf148-4803-45f4-a3ea-2b49fd16da72 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:21.168029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3dee88d8-f473-4b93-9af0-74cc982f218e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9ad8465e75f457aaa525394af05b11f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:21.171964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf640f-89df-4d6d-b7bd-b01e0d704a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:21.173226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf640f-89df-4d6d-b7bd-b01e0d704a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:21.173667 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf640f-89df-4d6d-b7bd-b01e0d704a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:21.173784 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf640f-89df-4d6d-b7bd-b01e0d704a8f admin admin] PATH_INFO: `/v3/projects/39f1e345c0a3416789513a519c804544` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:21.175683 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf640f-89df-4d6d-b7bd-b01e0d704a8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=39f1e345c0a3416789513a519c804544)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:21.178963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf640f-89df-4d6d-b7bd-b01e0d704a8f admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:21.191447 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3dee88d8-f473-4b93-9af0-74cc982f218e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9ad8465e75f457aaa525394af05b11f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:21.192140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3dee88d8-f473-4b93-9af0-74cc982f218e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9ad8465e75f457aaa525394af05b11f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:21.213200 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 707/1420] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:20 2022] DELETE /identity/v3/users/a9ad8465e75f457aaa525394af05b11f => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:21.214057 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee88d8-f473-4b93-9af0-74cc982f218e admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:21.219557 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3dee88d8-f473-4b93-9af0-74cc982f218e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:21.257854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-21bf640f-89df-4d6d-b7bd-b01e0d704a8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39f1e345c0a3416789513a519c804544', 'request_id': 'req-21bf640f-89df-4d6d-b7bd-b01e0d704a8f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:21.258969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:21.281368 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-21bf640f-89df-4d6d-b7bd-b01e0d704a8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39f1e345c0a3416789513a519c804544', 'request_id': 'req-21bf640f-89df-4d6d-b7bd-b01e0d704a8f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:21.302947 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-21bf640f-89df-4d6d-b7bd-b01e0d704a8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39f1e345c0a3416789513a519c804544', 'request_id': 'req-21bf640f-89df-4d6d-b7bd-b01e0d704a8f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:21.329391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-21bf640f-89df-4d6d-b7bd-b01e0d704a8f admin admin] The token cache is being invalidate because project 39f1e345c0a3416789513a519c804544 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:21.329688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-21bf640f-89df-4d6d-b7bd-b01e0d704a8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:21.331953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-21bf640f-89df-4d6d-b7bd-b01e0d704a8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:21.341913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 714/1421] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:21 2022] DELETE /identity/v3/projects/39f1e345c0a3416789513a519c804544 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:21.344856 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d292b-335d-43b0-bdf8-b08a62730e1c None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:21.347032 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d292b-335d-43b0-bdf8-b08a62730e1c None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:21.347317 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d292b-335d-43b0-bdf8-b08a62730e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:21.381573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-536d292b-335d-43b0-bdf8-b08a62730e1c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:21.384304 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-536d292b-335d-43b0-bdf8-b08a62730e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:21.401845 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee88d8-f473-4b93-9af0-74cc982f218e admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:21.504464 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 715/1422] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:21.509362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97e8f0-cfb0-4f29-82ea-80494b6ed5a2 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:21.509362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97e8f0-cfb0-4f29-82ea-80494b6ed5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:21.510003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97e8f0-cfb0-4f29-82ea-80494b6ed5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:21.515796 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2c691-bc8f-4f74-9dad-f69883a5d253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:21.517468 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2c691-bc8f-4f74-9dad-f69883a5d253 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:21.517992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2c691-bc8f-4f74-9dad-f69883a5d253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:21.518478 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2c691-bc8f-4f74-9dad-f69883a5d253 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:21.519690 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2c691-bc8f-4f74-9dad-f69883a5d253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:21.522815 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2c691-bc8f-4f74-9dad-f69883a5d253 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:21.556596 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-ae97e8f0-cfb0-4f29-82ea-80494b6ed5a2 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:21.557940 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ae97e8f0-cfb0-4f29-82ea-80494b6ed5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:21.574632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 708/1423] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:21 2022] POST /identity/v3/projects => generated 407 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:21.576124 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2c691-bc8f-4f74-9dad-f69883a5d253 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:21.578486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-00b2c691-bc8f-4f74-9dad-f69883a5d253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:21.580389 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 716/1424] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:21.581137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97e8f0-cfb0-4f29-82ea-80494b6ed5a2 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:21.588913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ae97e8f0-cfb0-4f29-82ea-80494b6ed5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:21.611064 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97e8f0-cfb0-4f29-82ea-80494b6ed5a2 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:21.623995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:21.757313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cef1b0-0c9c-4265-a0a7-48e9943a92fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:21.757884 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cef1b0-0c9c-4265-a0a7-48e9943a92fa admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:21.758164 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cef1b0-0c9c-4265-a0a7-48e9943a92fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:21.758432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cef1b0-0c9c-4265-a0a7-48e9943a92fa admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:21.759077 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cef1b0-0c9c-4265-a0a7-48e9943a92fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:21.761097 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cef1b0-0c9c-4265-a0a7-48e9943a92fa admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:21.765343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2c691-bc8f-4f74-9dad-f69883a5d253 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:21.781045 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c7813-7bc0-491c-9857-c828d5140f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:21.782199 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c7813-7bc0-491c-9857-c828d5140f84 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:21.782546 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c7813-7bc0-491c-9857-c828d5140f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:21.792408 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c7813-7bc0-491c-9857-c828d5140f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:21.807927 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 717/1425] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:21 2022] POST /identity/v3/users => generated 304 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:21.809704 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc166410-895d-4b9b-9267-85b15bc62a2e None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:21.811989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc166410-895d-4b9b-9267-85b15bc62a2e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:21.816280 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc166410-895d-4b9b-9267-85b15bc62a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:21.816939 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c7813-7bc0-491c-9857-c828d5140f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:21.818413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c7813-7bc0-491c-9857-c828d5140f84 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:21.850571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 709/1426] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:21.855279 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c7813-7bc0-491c-9857-c828d5140f84 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:21.856346 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-bc166410-895d-4b9b-9267-85b15bc62a2e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:21.858753 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-bc166410-895d-4b9b-9267-85b15bc62a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:21.872331 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 718/1427] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:21.873327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc166410-895d-4b9b-9267-85b15bc62a2e None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:21.887983 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c7813-7bc0-491c-9857-c828d5140f84 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:21.891803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc166410-895d-4b9b-9267-85b15bc62a2e None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:21.894707 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ab487-c3d8-4b63-af08-883834290ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:21.895950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ab487-c3d8-4b63-af08-883834290ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:21.896357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ab487-c3d8-4b63-af08-883834290ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:21.896644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ab487-c3d8-4b63-af08-883834290ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:21.897770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ab487-c3d8-4b63-af08-883834290ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:21.898340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d940da6-67f1-4c21-90ad-1645705ccdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:21.899115 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d940da6-67f1-4c21-90ad-1645705ccdfc admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:21.899546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d940da6-67f1-4c21-90ad-1645705ccdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:21.900172 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d940da6-67f1-4c21-90ad-1645705ccdfc admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:21.900584 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ab487-c3d8-4b63-af08-883834290ce0 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:21.901140 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d940da6-67f1-4c21-90ad-1645705ccdfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:21:21.902157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d940da6-67f1-4c21-90ad-1645705ccdfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:21.903966 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d940da6-67f1-4c21-90ad-1645705ccdfc admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:21.917956 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 719/1428] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:21:21.922381 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d940da6-67f1-4c21-90ad-1645705ccdfc admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:21.928591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 710/1429] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:21.931274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ab487-c3d8-4b63-af08-883834290ce0 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:21.962785 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d940da6-67f1-4c21-90ad-1645705ccdfc admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:21.963086 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ab487-c3d8-4b63-af08-883834290ce0 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:21.975807 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237036e-6a12-40b7-b47a-c5460a31432a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:21.976834 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237036e-6a12-40b7-b47a-c5460a31432a admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:21.977271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237036e-6a12-40b7-b47a-c5460a31432a admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:21.977850 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237036e-6a12-40b7-b47a-c5460a31432a admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:21.978959 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237036e-6a12-40b7-b47a-c5460a31432a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:21.979934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ce382-5397-4e0a-b88e-6bdfa744aa64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:21.980572 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ce382-5397-4e0a-b88e-6bdfa744aa64 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:21.980855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ce382-5397-4e0a-b88e-6bdfa744aa64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:21.983318 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ce382-5397-4e0a-b88e-6bdfa744aa64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:21.984028 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ce382-5397-4e0a-b88e-6bdfa744aa64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:21.987060 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ce382-5397-4e0a-b88e-6bdfa744aa64 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:21.987417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237036e-6a12-40b7-b47a-c5460a31432a admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:22.016432 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 711/1430] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:22.020259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ce382-5397-4e0a-b88e-6bdfa744aa64 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:22.042578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ce382-5397-4e0a-b88e-6bdfa744aa64 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:22.058145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5dbe7f-0722-48d6-a2dc-a97623723d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:22.061036 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dbe7f-0722-48d6-a2dc-a97623723d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:22.061918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dbe7f-0722-48d6-a2dc-a97623723d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:22.062222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dbe7f-0722-48d6-a2dc-a97623723d60 admin admin] PATH_INFO: `/v3/projects/4ecf4e8fc064433c95078468a33b1227/users/78e948ff9b12494dbbc752263b6c5405/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:22.082282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 720/1431] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:21 2022] POST /identity/v3/projects => generated 363 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:22.086611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237036e-6a12-40b7-b47a-c5460a31432a admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:22.110974 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5dbe7f-0722-48d6-a2dc-a97623723d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ecf4e8fc064433c95078468a33b1227, user_id=78e948ff9b12494dbbc752263b6c5405, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:22.114562 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5dbe7f-0722-48d6-a2dc-a97623723d60 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:22.137296 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:22.156620 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 712/1432] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:22 2022] PUT /identity/v3/projects/4ecf4e8fc064433c95078468a33b1227/users/78e948ff9b12494dbbc752263b6c5405/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:22.190922 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54244c1-9b84-40fe-b6c5-0321530bcc53 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:22.191393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54244c1-9b84-40fe-b6c5-0321530bcc53 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:22.191508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54244c1-9b84-40fe-b6c5-0321530bcc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:22.236008 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237036e-6a12-40b7-b47a-c5460a31432a admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:22.241683 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-c54244c1-9b84-40fe-b6c5-0321530bcc53 None None] MFA Rules not processed for user `78e948ff9b12494dbbc752263b6c5405`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:22.243120 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c54244c1-9b84-40fe-b6c5-0321530bcc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:22.326931 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35417d63-69e2-41d0-90a7-6df26d57ca27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:22.327970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35417d63-69e2-41d0-90a7-6df26d57ca27 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:22.328400 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35417d63-69e2-41d0-90a7-6df26d57ca27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:22.328843 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35417d63-69e2-41d0-90a7-6df26d57ca27 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:22.329499 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35417d63-69e2-41d0-90a7-6df26d57ca27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:22.332176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35417d63-69e2-41d0-90a7-6df26d57ca27 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:22.367521 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 721/1433] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:22 2022] POST /identity/v3/users => generated 282 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:22.369166 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35417d63-69e2-41d0-90a7-6df26d57ca27 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:22.377470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 713/1434] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:22 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:22.378482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54244c1-9b84-40fe-b6c5-0321530bcc53 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:22.385394 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35417d63-69e2-41d0-90a7-6df26d57ca27 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:22.393483 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43272738-f822-46d6-8f0d-bd8f6f199236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:22.394816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43272738-f822-46d6-8f0d-bd8f6f199236 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:22.395199 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43272738-f822-46d6-8f0d-bd8f6f199236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:22.395584 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43272738-f822-46d6-8f0d-bd8f6f199236 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:22.396941 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43272738-f822-46d6-8f0d-bd8f6f199236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:22.399437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43272738-f822-46d6-8f0d-bd8f6f199236 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:22.405490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54244c1-9b84-40fe-b6c5-0321530bcc53 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:22.412473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 722/1435] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:22.416385 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43272738-f822-46d6-8f0d-bd8f6f199236 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:22.428326 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55403a95-f8f5-4d73-833a-187f17f4960d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:22.429158 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55403a95-f8f5-4d73-833a-187f17f4960d admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:22.429594 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55403a95-f8f5-4d73-833a-187f17f4960d admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:22.430554 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55403a95-f8f5-4d73-833a-187f17f4960d admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:22.432427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55403a95-f8f5-4d73-833a-187f17f4960d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:22.433702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43272738-f822-46d6-8f0d-bd8f6f199236 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:22.436391 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55403a95-f8f5-4d73-833a-187f17f4960d admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:22.458767 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960eac14-6536-47ca-b3cf-a98b24ce82a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:22.459404 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960eac14-6536-47ca-b3cf-a98b24ce82a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:22.459718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960eac14-6536-47ca-b3cf-a98b24ce82a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:22.460060 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960eac14-6536-47ca-b3cf-a98b24ce82a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:22.460830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960eac14-6536-47ca-b3cf-a98b24ce82a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:22.462673 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960eac14-6536-47ca-b3cf-a98b24ce82a1 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:22.479169 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 723/1436] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:21:22.480303 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960eac14-6536-47ca-b3cf-a98b24ce82a1 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:22.494119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960eac14-6536-47ca-b3cf-a98b24ce82a1 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:22.497331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 714/1437] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:22 2022] POST /identity/v3/projects => generated 407 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:22.503836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55403a95-f8f5-4d73-833a-187f17f4960d admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:22.504208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff57dd-3ce4-4505-acac-ff21e162cc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:22.505140 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff57dd-3ce4-4505-acac-ff21e162cc9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:22.505788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff57dd-3ce4-4505-acac-ff21e162cc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:22.505788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff57dd-3ce4-4505-acac-ff21e162cc9d admin admin] PATH_INFO: `/v3/projects/086967943cb94332b5e7eee01e997c73/users/dd8459a660564c288f3ba82318d7318e/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:22.532333 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff57dd-3ce4-4505-acac-ff21e162cc9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=086967943cb94332b5e7eee01e997c73, user_id=dd8459a660564c288f3ba82318d7318e, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:22.537595 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff57dd-3ce4-4505-acac-ff21e162cc9d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:22.567010 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 724/1438] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:22 2022] PUT /identity/v3/projects/086967943cb94332b5e7eee01e997c73/users/dd8459a660564c288f3ba82318d7318e/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:22.578783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe65f19-6aec-4177-b8b6-c20a7f1e6fd2 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:22.579308 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe65f19-6aec-4177-b8b6-c20a7f1e6fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:22.579825 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe65f19-6aec-4177-b8b6-c20a7f1e6fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:22.582496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:22.637328 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-efe65f19-6aec-4177-b8b6-c20a7f1e6fd2 None None] MFA Rules not processed for user `dd8459a660564c288f3ba82318d7318e`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:22.638579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-efe65f19-6aec-4177-b8b6-c20a7f1e6fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:22.678637 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55403a95-f8f5-4d73-833a-187f17f4960d admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:22.730389 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:22.736857 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 725/1439] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:22.738028 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe65f19-6aec-4177-b8b6-c20a7f1e6fd2 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:22.845558 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe65f19-6aec-4177-b8b6-c20a7f1e6fd2 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:22.846273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075ce63-eb1f-431a-979a-2f2af33830db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:22.847281 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075ce63-eb1f-431a-979a-2f2af33830db admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:22.847747 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075ce63-eb1f-431a-979a-2f2af33830db admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:22.848166 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075ce63-eb1f-431a-979a-2f2af33830db admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:22.850370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075ce63-eb1f-431a-979a-2f2af33830db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:22.851397 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4961f70-d899-49d5-9a08-5f2d0204f4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:22.852331 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4961f70-d899-49d5-9a08-5f2d0204f4a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:22.852785 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4961f70-d899-49d5-9a08-5f2d0204f4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:22.853137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4961f70-d899-49d5-9a08-5f2d0204f4a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:22.854280 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4961f70-d899-49d5-9a08-5f2d0204f4a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:22.855614 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075ce63-eb1f-431a-979a-2f2af33830db admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:22.863805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4961f70-d899-49d5-9a08-5f2d0204f4a4 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:22.922006 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 726/1440] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:22 2022] POST /identity/v3/projects => generated 365 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:22.922780 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4961f70-d899-49d5-9a08-5f2d0204f4a4 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:22.924751 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 715/1441] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:22 2022] POST /identity/v3/users => generated 310 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:22.926473 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075ce63-eb1f-431a-979a-2f2af33830db admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:22.948856 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075ce63-eb1f-431a-979a-2f2af33830db admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:23.085475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4961f70-d899-49d5-9a08-5f2d0204f4a4 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:23.093111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e09dc7-c340-4fe4-898d-e734b58c5049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:23.094071 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e09dc7-c340-4fe4-898d-e734b58c5049 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:23.094497 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e09dc7-c340-4fe4-898d-e734b58c5049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:23.095281 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e09dc7-c340-4fe4-898d-e734b58c5049 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:23.096318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e09dc7-c340-4fe4-898d-e734b58c5049 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:23.099056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e09dc7-c340-4fe4-898d-e734b58c5049 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:23.110836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0affeef-caf6-4d2d-a956-8752872e1ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:23.112137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0affeef-caf6-4d2d-a956-8752872e1ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:23.113234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0affeef-caf6-4d2d-a956-8752872e1ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:23.113524 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0affeef-caf6-4d2d-a956-8752872e1ab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:23.114701 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0affeef-caf6-4d2d-a956-8752872e1ab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:23.116700 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0affeef-caf6-4d2d-a956-8752872e1ab6 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:23.133250 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 727/1442] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:22 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:23.134686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e09dc7-c340-4fe4-898d-e734b58c5049 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:23.135756 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 716/1443] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:23.136870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0affeef-caf6-4d2d-a956-8752872e1ab6 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:23.151387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0affeef-caf6-4d2d-a956-8752872e1ab6 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:23.159150 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce0a1f-69d0-4b4d-9300-d316b34f77b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:23.161566 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce0a1f-69d0-4b4d-9300-d316b34f77b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:23.161850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce0a1f-69d0-4b4d-9300-d316b34f77b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:23.163068 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce0a1f-69d0-4b4d-9300-d316b34f77b5 admin admin] PATH_INFO: `/v3/projects/3c27c5ae7e3d477996c064ad6b208329/users/58ea4271935c4c468543ae55d0e53426/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:23.166950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e09dc7-c340-4fe4-898d-e734b58c5049 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:23.181169 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea599eee-1eaa-4ec2-a29f-c4a19cb4493b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:23.181840 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea599eee-1eaa-4ec2-a29f-c4a19cb4493b admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:23.182111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea599eee-1eaa-4ec2-a29f-c4a19cb4493b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:23.182390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea599eee-1eaa-4ec2-a29f-c4a19cb4493b admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:23.183146 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea599eee-1eaa-4ec2-a29f-c4a19cb4493b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:23.184920 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea599eee-1eaa-4ec2-a29f-c4a19cb4493b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:23.214904 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 728/1444] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:21:23.216131 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea599eee-1eaa-4ec2-a29f-c4a19cb4493b admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:23.223848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ea599eee-1eaa-4ec2-a29f-c4a19cb4493b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:23.231007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce0a1f-69d0-4b4d-9300-d316b34f77b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c27c5ae7e3d477996c064ad6b208329, user_id=58ea4271935c4c468543ae55d0e53426, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:23.240274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce0a1f-69d0-4b4d-9300-d316b34f77b5 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:23.250195 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea599eee-1eaa-4ec2-a29f-c4a19cb4493b admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:23.259557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784205ca-6e29-4528-a908-4b82688cdaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:23.260358 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784205ca-6e29-4528-a908-4b82688cdaa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:23.260650 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784205ca-6e29-4528-a908-4b82688cdaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:23.260929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784205ca-6e29-4528-a908-4b82688cdaa8 admin admin] PATH_INFO: `/v3/projects/4cc9ad8fcdb44df1a65edc9fb7f31e92` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:23.264899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 717/1445] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:23 2022] PUT /identity/v3/projects/3c27c5ae7e3d477996c064ad6b208329/users/58ea4271935c4c468543ae55d0e53426/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:23.269357 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784205ca-6e29-4528-a908-4b82688cdaa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cc9ad8fcdb44df1a65edc9fb7f31e92)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:23.270106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce0a1f-69d0-4b4d-9300-d316b34f77b5 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:23.274534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784205ca-6e29-4528-a908-4b82688cdaa8 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:23.393429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:23.413656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-784205ca-6e29-4528-a908-4b82688cdaa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cc9ad8fcdb44df1a65edc9fb7f31e92', 'request_id': 'req-784205ca-6e29-4528-a908-4b82688cdaa8'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:23.452059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-784205ca-6e29-4528-a908-4b82688cdaa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cc9ad8fcdb44df1a65edc9fb7f31e92', 'request_id': 'req-784205ca-6e29-4528-a908-4b82688cdaa8'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:23.460201 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:23.465260 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce0a1f-69d0-4b4d-9300-d316b34f77b5 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:23.468063 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-784205ca-6e29-4528-a908-4b82688cdaa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cc9ad8fcdb44df1a65edc9fb7f31e92', 'request_id': 'req-784205ca-6e29-4528-a908-4b82688cdaa8'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:23.496802 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-784205ca-6e29-4528-a908-4b82688cdaa8 admin admin] The token cache is being invalidate because project 4cc9ad8fcdb44df1a65edc9fb7f31e92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:23.497089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-784205ca-6e29-4528-a908-4b82688cdaa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:23.498902 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-784205ca-6e29-4528-a908-4b82688cdaa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:23.500739 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 729/1446] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:23 2022] DELETE /identity/v3/projects/4cc9ad8fcdb44df1a65edc9fb7f31e92 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:23.504059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784205ca-6e29-4528-a908-4b82688cdaa8 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:23.504821 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-784205ca-6e29-4528-a908-4b82688cdaa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:23.614111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784205ca-6e29-4528-a908-4b82688cdaa8 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:23.619197 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80651fb6-6e29-452a-87f0-0bc46f0b5560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:23.619846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80651fb6-6e29-452a-87f0-0bc46f0b5560 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:23.620129 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80651fb6-6e29-452a-87f0-0bc46f0b5560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:23.620401 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80651fb6-6e29-452a-87f0-0bc46f0b5560 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:23.621165 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80651fb6-6e29-452a-87f0-0bc46f0b5560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:23.622966 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80651fb6-6e29-452a-87f0-0bc46f0b5560 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:23.625089 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75fd7b-5f91-4d36-97a9-7d197ff15acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:23.625864 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75fd7b-5f91-4d36-97a9-7d197ff15acb admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:23.626253 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75fd7b-5f91-4d36-97a9-7d197ff15acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:23.627039 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75fd7b-5f91-4d36-97a9-7d197ff15acb admin admin] PATH_INFO: `/v3/projects/f222279eb00a489da47816ea31c0edf8/users/117bf14d7fba4430ae2f058ca3c29667/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:23.645501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c75fd7b-5f91-4d36-97a9-7d197ff15acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f222279eb00a489da47816ea31c0edf8, user_id=117bf14d7fba4430ae2f058ca3c29667, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:23.646386 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 730/1447] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:23.649171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c75fd7b-5f91-4d36-97a9-7d197ff15acb admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:23.656431 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac77d8-ab88-43dc-bc3f-88b4afcbf242 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:23.656969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac77d8-ab88-43dc-bc3f-88b4afcbf242 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:23.657396 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac77d8-ab88-43dc-bc3f-88b4afcbf242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:23.670306 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 718/1448] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:23 2022] PUT /identity/v3/projects/f222279eb00a489da47816ea31c0edf8/users/117bf14d7fba4430ae2f058ca3c29667/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:23.671287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75fd7b-5f91-4d36-97a9-7d197ff15acb admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:23.686858 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-ccac77d8-ab88-43dc-bc3f-88b4afcbf242 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:23.688590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ccac77d8-ab88-43dc-bc3f-88b4afcbf242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:23.770252 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 731/1449] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:23.771294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac77d8-ab88-43dc-bc3f-88b4afcbf242 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:23.773072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ccac77d8-ab88-43dc-bc3f-88b4afcbf242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:23.779251 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75fd7b-5f91-4d36-97a9-7d197ff15acb admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:23.784893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d35084-e73d-43a4-a367-8f06baa270e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:23.785597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d35084-e73d-43a4-a367-8f06baa270e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:23.785858 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac77d8-ab88-43dc-bc3f-88b4afcbf242 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:23.785966 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d35084-e73d-43a4-a367-8f06baa270e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:23.786143 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d35084-e73d-43a4-a367-8f06baa270e1 admin admin] PATH_INFO: `/v3/projects/3c27c5ae7e3d477996c064ad6b208329/users/58ea4271935c4c468543ae55d0e53426/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:23.788182 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d35084-e73d-43a4-a367-8f06baa270e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c27c5ae7e3d477996c064ad6b208329, user_id=58ea4271935c4c468543ae55d0e53426, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:23.790911 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d35084-e73d-43a4-a367-8f06baa270e1 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:23.791916 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec132d-97b0-4f55-adad-5847a1e1badf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:23.792531 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec132d-97b0-4f55-adad-5847a1e1badf admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:23.792865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec132d-97b0-4f55-adad-5847a1e1badf admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:23.793146 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec132d-97b0-4f55-adad-5847a1e1badf admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:23.793939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec132d-97b0-4f55-adad-5847a1e1badf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:23.795610 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec132d-97b0-4f55-adad-5847a1e1badf admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:23.803477 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 719/1450] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:23 2022] PUT /identity/v3/projects/3c27c5ae7e3d477996c064ad6b208329/users/58ea4271935c4c468543ae55d0e53426/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:23.804987 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df83d5-97de-4cce-a6f1-1601ab31cde6 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:23.805276 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df83d5-97de-4cce-a6f1-1601ab31cde6 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:23.805592 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df83d5-97de-4cce-a6f1-1601ab31cde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:23.815378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 732/1451] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:23.816665 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83d52d-0913-49ce-8345-373ee962756d None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:23.817020 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83d52d-0913-49ce-8345-373ee962756d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:23.817364 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83d52d-0913-49ce-8345-373ee962756d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:23.837078 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-63df83d5-97de-4cce-a6f1-1601ab31cde6 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:23.838627 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-63df83d5-97de-4cce-a6f1-1601ab31cde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:23.846703 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-8c83d52d-0913-49ce-8345-373ee962756d None None] MFA Rules not processed for user `58ea4271935c4c468543ae55d0e53426`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:23.847825 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8c83d52d-0913-49ce-8345-373ee962756d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:23.905507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 720/1452] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:23.906505 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df83d5-97de-4cce-a6f1-1601ab31cde6 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:23.911910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:23.918101 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df83d5-97de-4cce-a6f1-1601ab31cde6 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:23.923890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba598c5-66f4-4d78-ac2b-e959c9b4f743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:23.924550 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba598c5-66f4-4d78-ac2b-e959c9b4f743 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:23.924830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba598c5-66f4-4d78-ac2b-e959c9b4f743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:23.925779 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba598c5-66f4-4d78-ac2b-e959c9b4f743 admin admin] PATH_INFO: `/v3/projects/f222279eb00a489da47816ea31c0edf8/users/117bf14d7fba4430ae2f058ca3c29667/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:23.927311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:23.928180 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba598c5-66f4-4d78-ac2b-e959c9b4f743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f222279eb00a489da47816ea31c0edf8, user_id=117bf14d7fba4430ae2f058ca3c29667, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:23.931067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba598c5-66f4-4d78-ac2b-e959c9b4f743 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:23.935868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 733/1453] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:23 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:23.936885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83d52d-0913-49ce-8345-373ee962756d None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:23.944490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 721/1454] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:23 2022] PUT /identity/v3/projects/f222279eb00a489da47816ea31c0edf8/users/117bf14d7fba4430ae2f058ca3c29667/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:23.945330 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba598c5-66f4-4d78-ac2b-e959c9b4f743 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:23.947939 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-7ba598c5-66f4-4d78-ac2b-e959c9b4f743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:23.951906 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83d52d-0913-49ce-8345-373ee962756d None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:24.014656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba598c5-66f4-4d78-ac2b-e959c9b4f743 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:24.019733 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db651129-4683-410c-be76-cd565004f06d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:24.020445 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db651129-4683-410c-be76-cd565004f06d service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:24.020841 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db651129-4683-410c-be76-cd565004f06d service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:24.021114 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db651129-4683-410c-be76-cd565004f06d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:24.021262 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff14cf-b87d-4e9e-a9f4-e9d492ca7fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:24.021998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff14cf-b87d-4e9e-a9f4-e9d492ca7fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:24.022612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff14cf-b87d-4e9e-a9f4-e9d492ca7fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:24.023021 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff14cf-b87d-4e9e-a9f4-e9d492ca7fe2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:24.023769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff14cf-b87d-4e9e-a9f4-e9d492ca7fe2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:21:24.024143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff14cf-b87d-4e9e-a9f4-e9d492ca7fe2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:24.025839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff14cf-b87d-4e9e-a9f4-e9d492ca7fe2 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:24.052211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 734/1455] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:21:24.053649 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094b343-d5e0-4bf0-a8a7-75f61668ca85 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:24.054246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094b343-d5e0-4bf0-a8a7-75f61668ca85 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:24.054599 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094b343-d5e0-4bf0-a8a7-75f61668ca85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:24.087297 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-f094b343-d5e0-4bf0-a8a7-75f61668ca85 None None] MFA Rules not processed for user `117bf14d7fba4430ae2f058ca3c29667`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:24.088739 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f094b343-d5e0-4bf0-a8a7-75f61668ca85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:24.100526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db651129-4683-410c-be76-cd565004f06d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:24.102164 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db651129-4683-410c-be76-cd565004f06d service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:24.113055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 722/1456] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:23 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:24.114080 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db651129-4683-410c-be76-cd565004f06d service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:24.125674 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db651129-4683-410c-be76-cd565004f06d service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:24.131625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a210a9-5eca-484e-a1aa-f653d2204669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:24.132424 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a210a9-5eca-484e-a1aa-f653d2204669 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:24.132849 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a210a9-5eca-484e-a1aa-f653d2204669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:24.133307 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a210a9-5eca-484e-a1aa-f653d2204669 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:24.134265 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a210a9-5eca-484e-a1aa-f653d2204669 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:24.136704 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a210a9-5eca-484e-a1aa-f653d2204669 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:24.192413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 723/1457] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:24 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:24.196152 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a210a9-5eca-484e-a1aa-f653d2204669 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:24.292473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 735/1458] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:24.302775 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094b343-d5e0-4bf0-a8a7-75f61668ca85 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:24.343460 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094b343-d5e0-4bf0-a8a7-75f61668ca85 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:24.430174 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a210a9-5eca-484e-a1aa-f653d2204669 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:24.443277 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1246887-8a78-40af-9b48-16bfaf62db8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:24.444516 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1246887-8a78-40af-9b48-16bfaf62db8a admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:24.444819 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1246887-8a78-40af-9b48-16bfaf62db8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:24.445705 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1246887-8a78-40af-9b48-16bfaf62db8a admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:24.447974 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1246887-8a78-40af-9b48-16bfaf62db8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:24.453302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1246887-8a78-40af-9b48-16bfaf62db8a admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:24.506132 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 724/1459] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:24 2022] POST /identity/v3/users => generated 286 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:24.509241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1246887-8a78-40af-9b48-16bfaf62db8a admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:24.526827 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94634e3-c6e7-42ba-bbc3-c6a554f2a351 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:24.527473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94634e3-c6e7-42ba-bbc3-c6a554f2a351 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:24.527990 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94634e3-c6e7-42ba-bbc3-c6a554f2a351 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:24.528098 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94634e3-c6e7-42ba-bbc3-c6a554f2a351 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:24.543496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1246887-8a78-40af-9b48-16bfaf62db8a admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:24.556803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bcc86a-9e20-4606-aee0-c2b801580ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:24.557479 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcc86a-9e20-4606-aee0-c2b801580ccc admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:24.557758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcc86a-9e20-4606-aee0-c2b801580ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:24.558028 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcc86a-9e20-4606-aee0-c2b801580ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:24.558670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bcc86a-9e20-4606-aee0-c2b801580ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:24.560281 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bcc86a-9e20-4606-aee0-c2b801580ccc admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:24.570714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:24.597370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 725/1460] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:24.598599 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bcc86a-9e20-4606-aee0-c2b801580ccc admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:24.610444 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:24.625945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bcc86a-9e20-4606-aee0-c2b801580ccc admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:24.635771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2df9b6-6d0c-4a3c-95a3-02b530a86d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:24.636396 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2df9b6-6d0c-4a3c-95a3-02b530a86d1f admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:24.636669 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2df9b6-6d0c-4a3c-95a3-02b530a86d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:24.636935 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2df9b6-6d0c-4a3c-95a3-02b530a86d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:24.637798 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2df9b6-6d0c-4a3c-95a3-02b530a86d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:24.639388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2df9b6-6d0c-4a3c-95a3-02b530a86d1f admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:24.665415 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 726/1461] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:24.665415 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2df9b6-6d0c-4a3c-95a3-02b530a86d1f admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:24.672490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-bd2df9b6-6d0c-4a3c-95a3-02b530a86d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:24.676947 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94634e3-c6e7-42ba-bbc3-c6a554f2a351 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:24.679152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94634e3-c6e7-42ba-bbc3-c6a554f2a351 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:24.694603 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 736/1462] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:24.695439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94634e3-c6e7-42ba-bbc3-c6a554f2a351 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:24.705765 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2df9b6-6d0c-4a3c-95a3-02b530a86d1f admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:24.707348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94634e3-c6e7-42ba-bbc3-c6a554f2a351 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:24.714406 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424cd38-276a-485d-835f-b5eb01ce686c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:24.716479 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e6880-f900-4fec-90ec-b3f176a3556d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:24.717397 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e6880-f900-4fec-90ec-b3f176a3556d service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:24.720215 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e6880-f900-4fec-90ec-b3f176a3556d service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:24.720696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e6880-f900-4fec-90ec-b3f176a3556d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:24.724427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-5e9e6880-f900-4fec-90ec-b3f176a3556d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:24.758008 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424cd38-276a-485d-835f-b5eb01ce686c admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:24.758956 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424cd38-276a-485d-835f-b5eb01ce686c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:24.759405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424cd38-276a-485d-835f-b5eb01ce686c admin admin] PATH_INFO: `/v3/projects/acd90093a8f84838b9246d471c278f3e/users/b0f11f793eef43be8002fec05dd78840/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:24.874924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e6880-f900-4fec-90ec-b3f176a3556d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:24.877189 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e6880-f900-4fec-90ec-b3f176a3556d service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:24.886539 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424cd38-276a-485d-835f-b5eb01ce686c admin admin] RBAC: Authorizing `identity:create_grant(project_id=acd90093a8f84838b9246d471c278f3e, user_id=b0f11f793eef43be8002fec05dd78840, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:24.889640 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424cd38-276a-485d-835f-b5eb01ce686c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:24.912278 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 727/1463] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:24 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:24.914324 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e6880-f900-4fec-90ec-b3f176a3556d service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:24.943606 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 737/1464] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:24 2022] PUT /identity/v3/projects/acd90093a8f84838b9246d471c278f3e/users/b0f11f793eef43be8002fec05dd78840/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:24.944150 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e6880-f900-4fec-90ec-b3f176a3556d service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:24.950356 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1d54f-37d0-498f-8a43-7018988ffa89 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:24.950660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1d54f-37d0-498f-8a43-7018988ffa89 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:24.952588 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1d54f-37d0-498f-8a43-7018988ffa89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:25.040301 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-22c1d54f-37d0-498f-8a43-7018988ffa89 None None] MFA Rules not processed for user `b0f11f793eef43be8002fec05dd78840`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:25.043689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-22c1d54f-37d0-498f-8a43-7018988ffa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:25.064459 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49320132-57cd-42ee-9ec9-6ccb8c6ac12e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:25.065252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49320132-57cd-42ee-9ec9-6ccb8c6ac12e service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:25.066143 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49320132-57cd-42ee-9ec9-6ccb8c6ac12e service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:25.066495 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49320132-57cd-42ee-9ec9-6ccb8c6ac12e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:25.068495 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-49320132-57cd-42ee-9ec9-6ccb8c6ac12e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:25.152615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 738/1465] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:25.157553 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1d54f-37d0-498f-8a43-7018988ffa89 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:25.168662 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49320132-57cd-42ee-9ec9-6ccb8c6ac12e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:25.170050 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49320132-57cd-42ee-9ec9-6ccb8c6ac12e service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:25.185485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 728/1466] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:25.244098 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1d54f-37d0-498f-8a43-7018988ffa89 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:25.248939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e0b4c-fb23-4ae4-87f9-ff496345f118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:25.249639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e0b4c-fb23-4ae4-87f9-ff496345f118 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:25.249920 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e0b4c-fb23-4ae4-87f9-ff496345f118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:25.250185 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e0b4c-fb23-4ae4-87f9-ff496345f118 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:25.250815 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e0b4c-fb23-4ae4-87f9-ff496345f118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:25.252778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e0b4c-fb23-4ae4-87f9-ff496345f118 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:25.286139 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 739/1467] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:25 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:21:25.290251 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49320132-57cd-42ee-9ec9-6ccb8c6ac12e service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:25.311095 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49320132-57cd-42ee-9ec9-6ccb8c6ac12e service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:25.358056 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:25.385652 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497839af-06c9-4fea-b986-153c2b480120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:25.386274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497839af-06c9-4fea-b986-153c2b480120 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:25.386559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497839af-06c9-4fea-b986-153c2b480120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:25.386917 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497839af-06c9-4fea-b986-153c2b480120 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:25.387461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497839af-06c9-4fea-b986-153c2b480120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:25.390892 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497839af-06c9-4fea-b986-153c2b480120 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:25.419688 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 729/1468] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:25 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:25.420875 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e0b4c-fb23-4ae4-87f9-ff496345f118 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:25.434612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e0b4c-fb23-4ae4-87f9-ff496345f118 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:25.440188 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1b1e4-fd32-4d58-9c10-9380454a88c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:25.440988 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1b1e4-fd32-4d58-9c10-9380454a88c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:25.441447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1b1e4-fd32-4d58-9c10-9380454a88c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:25.441901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1b1e4-fd32-4d58-9c10-9380454a88c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:25.443391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1b1e4-fd32-4d58-9c10-9380454a88c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:25.445537 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1b1e4-fd32-4d58-9c10-9380454a88c3 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:25.460059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 740/1469] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:25.461207 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497839af-06c9-4fea-b986-153c2b480120 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:25.471981 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497839af-06c9-4fea-b986-153c2b480120 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:25.477433 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174fa9c-2acf-42b9-a782-d33155a507e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:25.478099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174fa9c-2acf-42b9-a782-d33155a507e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:25.478377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174fa9c-2acf-42b9-a782-d33155a507e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:25.478649 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174fa9c-2acf-42b9-a782-d33155a507e9 admin admin] PATH_INFO: `/v3/projects/477e091cf048486990294de8f4153f29/users/d2d971da66114c76a270fd77f1ca9924/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:25.496535 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174fa9c-2acf-42b9-a782-d33155a507e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=477e091cf048486990294de8f4153f29, user_id=d2d971da66114c76a270fd77f1ca9924, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:25.499202 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174fa9c-2acf-42b9-a782-d33155a507e9 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:25.513452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 730/1470] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:25 2022] PUT /identity/v3/projects/477e091cf048486990294de8f4153f29/users/d2d971da66114c76a270fd77f1ca9924/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:25.517302 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1b1e4-fd32-4d58-9c10-9380454a88c3 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:25.528053 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1b1e4-fd32-4d58-9c10-9380454a88c3 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:25.532338 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:25.586501 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7df4d8-ecfa-49c2-83a2-84d0e70864f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:25.587131 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7df4d8-ecfa-49c2-83a2-84d0e70864f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:25.587410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7df4d8-ecfa-49c2-83a2-84d0e70864f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:25.587718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7df4d8-ecfa-49c2-83a2-84d0e70864f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:25.588434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7df4d8-ecfa-49c2-83a2-84d0e70864f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:25.590046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7df4d8-ecfa-49c2-83a2-84d0e70864f2 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:25.599916 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 741/1471] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:21:25.604090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174fa9c-2acf-42b9-a782-d33155a507e9 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:25.614270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174fa9c-2acf-42b9-a782-d33155a507e9 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:25.618746 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa948b33-8bda-4c59-93e8-eced34e3eae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:25.619389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa948b33-8bda-4c59-93e8-eced34e3eae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:25.619691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa948b33-8bda-4c59-93e8-eced34e3eae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:25.619963 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa948b33-8bda-4c59-93e8-eced34e3eae3 admin admin] PATH_INFO: `/v3/projects/477e091cf048486990294de8f4153f29/users/d2d971da66114c76a270fd77f1ca9924/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:25.621831 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa948b33-8bda-4c59-93e8-eced34e3eae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=477e091cf048486990294de8f4153f29, user_id=d2d971da66114c76a270fd77f1ca9924, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:25.624474 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa948b33-8bda-4c59-93e8-eced34e3eae3 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:25.634816 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 731/1472] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:25 2022] PUT /identity/v3/projects/477e091cf048486990294de8f4153f29/users/d2d971da66114c76a270fd77f1ca9924/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:25.645414 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337d2ae-9e64-4ddc-9207-9ad3754ffac4 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:25.645698 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337d2ae-9e64-4ddc-9207-9ad3754ffac4 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:25.645969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337d2ae-9e64-4ddc-9207-9ad3754ffac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:25.675551 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-9337d2ae-9e64-4ddc-9207-9ad3754ffac4 None None] MFA Rules not processed for user `d2d971da66114c76a270fd77f1ca9924`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:25.676801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9337d2ae-9e64-4ddc-9207-9ad3754ffac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:25.745998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 742/1473] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:25.758247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa948b33-8bda-4c59-93e8-eced34e3eae3 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:25.815163 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa948b33-8bda-4c59-93e8-eced34e3eae3 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:25.819304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2ee42-172e-406b-907f-c55486e0f13f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:25.819960 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2ee42-172e-406b-907f-c55486e0f13f service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:25.820227 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2ee42-172e-406b-907f-c55486e0f13f service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:25.820498 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2ee42-172e-406b-907f-c55486e0f13f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:25.829362 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2ee42-172e-406b-907f-c55486e0f13f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:25.830781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2ee42-172e-406b-907f-c55486e0f13f service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:25.840435 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 732/1474] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:26.261301 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337d2ae-9e64-4ddc-9207-9ad3754ffac4 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:26.330068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337d2ae-9e64-4ddc-9207-9ad3754ffac4 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:26.334654 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb98547-ad8e-41dc-9a64-fa6b477a73ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:26.335291 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb98547-ad8e-41dc-9a64-fa6b477a73ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:26.335643 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb98547-ad8e-41dc-9a64-fa6b477a73ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:26.335887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb98547-ad8e-41dc-9a64-fa6b477a73ea admin admin] PATH_INFO: `/v3/users/b0f11f793eef43be8002fec05dd78840` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:26.337008 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb98547-ad8e-41dc-9a64-fa6b477a73ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0f11f793eef43be8002fec05dd78840)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:26.338952 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb98547-ad8e-41dc-9a64-fa6b477a73ea admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:26.373893 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:26.388609 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-3bb98547-ad8e-41dc-9a64-fa6b477a73ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0f11f793eef43be8002fec05dd78840'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:26.414314 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-3bb98547-ad8e-41dc-9a64-fa6b477a73ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0f11f793eef43be8002fec05dd78840'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:26.421976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-3bb98547-ad8e-41dc-9a64-fa6b477a73ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0f11f793eef43be8002fec05dd78840'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:26.422781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-3bb98547-ad8e-41dc-9a64-fa6b477a73ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0f11f793eef43be8002fec05dd78840'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:26.430201 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-3bb98547-ad8e-41dc-9a64-fa6b477a73ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0f11f793eef43be8002fec05dd78840'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:26.441396 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 743/1475] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:26 2022] DELETE /identity/v3/users/b0f11f793eef43be8002fec05dd78840 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:26.443012 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cde10b-5f03-4912-aae1-0368bc5a27e2 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:26.443354 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cde10b-5f03-4912-aae1-0368bc5a27e2 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:26.443588 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cde10b-5f03-4912-aae1-0368bc5a27e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:26.476757 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-77cde10b-5f03-4912-aae1-0368bc5a27e2 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:26.478004 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-77cde10b-5f03-4912-aae1-0368bc5a27e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:26.596505 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 733/1476] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:26.603694 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb98547-ad8e-41dc-9a64-fa6b477a73ea admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:26.605809 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-3bb98547-ad8e-41dc-9a64-fa6b477a73ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:26.710010 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb98547-ad8e-41dc-9a64-fa6b477a73ea admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:26.722242 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55dc5b2-603b-4ad0-92b9-c0f2593d225d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:26.728091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dc5b2-603b-4ad0-92b9-c0f2593d225d service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:26.728668 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dc5b2-603b-4ad0-92b9-c0f2593d225d service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:26.729129 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dc5b2-603b-4ad0-92b9-c0f2593d225d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:26.742937 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55dc5b2-603b-4ad0-92b9-c0f2593d225d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:26.744350 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55dc5b2-603b-4ad0-92b9-c0f2593d225d service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:26.764355 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 744/1477] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:27.042331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cde10b-5f03-4912-aae1-0368bc5a27e2 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:27.043858 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-77cde10b-5f03-4912-aae1-0368bc5a27e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:27.059836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cde10b-5f03-4912-aae1-0368bc5a27e2 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:27.063450 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:27.064296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:27.064647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:27.064995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820 admin admin] PATH_INFO: `/v3/projects/acd90093a8f84838b9246d471c278f3e` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:27.066257 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acd90093a8f84838b9246d471c278f3e)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:27.068282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:27.104251 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:27.121595 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acd90093a8f84838b9246d471c278f3e', 'request_id': 'req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:27.132797 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acd90093a8f84838b9246d471c278f3e', 'request_id': 'req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:27.139858 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acd90093a8f84838b9246d471c278f3e', 'request_id': 'req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:27.153131 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820 admin admin] The token cache is being invalidate because project acd90093a8f84838b9246d471c278f3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:27.153491 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:27.154098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:27.155659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 734/1478] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:27 2022] DELETE /identity/v3/projects/acd90093a8f84838b9246d471c278f3e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:27.160506 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55dc5b2-603b-4ad0-92b9-c0f2593d225d service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:27.170861 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55dc5b2-603b-4ad0-92b9-c0f2593d225d service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:27.231653 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b1db7-d482-4a3a-92b7-52f36f4c6a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:27.232222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b1db7-d482-4a3a-92b7-52f36f4c6a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:27.232498 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b1db7-d482-4a3a-92b7-52f36f4c6a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:27.232768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b1db7-d482-4a3a-92b7-52f36f4c6a21 admin admin] PATH_INFO: `/v3/users/d2d971da66114c76a270fd77f1ca9924` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:27.234132 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b1db7-d482-4a3a-92b7-52f36f4c6a21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2d971da66114c76a270fd77f1ca9924)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:27.238324 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b1db7-d482-4a3a-92b7-52f36f4c6a21 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:27.298564 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c49b1db7-d482-4a3a-92b7-52f36f4c6a21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2d971da66114c76a270fd77f1ca9924'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:27.304765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:27.314564 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c49b1db7-d482-4a3a-92b7-52f36f4c6a21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2d971da66114c76a270fd77f1ca9924'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:27.325829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c49b1db7-d482-4a3a-92b7-52f36f4c6a21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2d971da66114c76a270fd77f1ca9924'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:27.326621 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c49b1db7-d482-4a3a-92b7-52f36f4c6a21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2d971da66114c76a270fd77f1ca9924'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:27.334770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c49b1db7-d482-4a3a-92b7-52f36f4c6a21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2d971da66114c76a270fd77f1ca9924'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:27.348661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 745/1479] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:27 2022] DELETE /identity/v3/users/d2d971da66114c76a270fd77f1ca9924 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:27.668211 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:27.669616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:27.685550 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6ff7f-b30e-4bbe-967c-0d9315fab820 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:27.753862 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db06eeb-3f74-44c3-bb02-56490c507700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:27.754443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db06eeb-3f74-44c3-bb02-56490c507700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:27.754724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db06eeb-3f74-44c3-bb02-56490c507700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:27.754996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db06eeb-3f74-44c3-bb02-56490c507700 admin admin] PATH_INFO: `/v3/projects/477e091cf048486990294de8f4153f29` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:27.756164 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db06eeb-3f74-44c3-bb02-56490c507700 admin admin] RBAC: Authorizing `identity:delete_project(project_id=477e091cf048486990294de8f4153f29)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:27.758100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db06eeb-3f74-44c3-bb02-56490c507700 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:27.805616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-8db06eeb-3f74-44c3-bb02-56490c507700 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '477e091cf048486990294de8f4153f29', 'request_id': 'req-8db06eeb-3f74-44c3-bb02-56490c507700'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:27.813544 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-8db06eeb-3f74-44c3-bb02-56490c507700 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '477e091cf048486990294de8f4153f29', 'request_id': 'req-8db06eeb-3f74-44c3-bb02-56490c507700'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:27.820338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-8db06eeb-3f74-44c3-bb02-56490c507700 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '477e091cf048486990294de8f4153f29', 'request_id': 'req-8db06eeb-3f74-44c3-bb02-56490c507700'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:27.831763 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-8db06eeb-3f74-44c3-bb02-56490c507700 admin admin] The token cache is being invalidate because project 477e091cf048486990294de8f4153f29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:27.832053 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-8db06eeb-3f74-44c3-bb02-56490c507700 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:27.832727 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-8db06eeb-3f74-44c3-bb02-56490c507700 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:27.834505 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 735/1480] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:27 2022] DELETE /identity/v3/projects/477e091cf048486990294de8f4153f29 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:27.839084 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af117cfb-f22d-454a-b514-49eeb2eeaf8c None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:27.839549 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af117cfb-f22d-454a-b514-49eeb2eeaf8c None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:27.839736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af117cfb-f22d-454a-b514-49eeb2eeaf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:27.859463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-af117cfb-f22d-454a-b514-49eeb2eeaf8c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:27.860700 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-af117cfb-f22d-454a-b514-49eeb2eeaf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:27.920423 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 746/1481] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:27.928594 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa86e74-d128-4dfc-98ca-472f6ae802d5 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:27.928883 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa86e74-d128-4dfc-98ca-472f6ae802d5 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:27.929158 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa86e74-d128-4dfc-98ca-472f6ae802d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:27.949293 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-9aa86e74-d128-4dfc-98ca-472f6ae802d5 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:27.950486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9aa86e74-d128-4dfc-98ca-472f6ae802d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:27.958957 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 736/1482] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:27.963116 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af117cfb-f22d-454a-b514-49eeb2eeaf8c None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:27.972688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af117cfb-f22d-454a-b514-49eeb2eeaf8c None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:27.977214 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a053d6c-0131-43f0-a9ec-8db09eeb6e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:27.977802 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a053d6c-0131-43f0-a9ec-8db09eeb6e5d admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:27.978077 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a053d6c-0131-43f0-a9ec-8db09eeb6e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:27.978347 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a053d6c-0131-43f0-a9ec-8db09eeb6e5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:27.978977 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a053d6c-0131-43f0-a9ec-8db09eeb6e5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:21:27.979277 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a053d6c-0131-43f0-a9ec-8db09eeb6e5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:27.980901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a053d6c-0131-43f0-a9ec-8db09eeb6e5d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:27.995694 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 747/1483] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:21:27.997195 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa86e74-d128-4dfc-98ca-472f6ae802d5 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:28.006972 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa86e74-d128-4dfc-98ca-472f6ae802d5 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:28.011857 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc70ac-608c-469e-ac17-ab3b56d993c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:28.012158 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc70ac-608c-469e-ac17-ab3b56d993c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:28.012414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc70ac-608c-469e-ac17-ab3b56d993c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:28.012685 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc70ac-608c-469e-ac17-ab3b56d993c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:28.013312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc70ac-608c-469e-ac17-ab3b56d993c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:28.015227 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc70ac-608c-469e-ac17-ab3b56d993c5 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:28.047687 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 737/1484] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:27 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:28.049742 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a053d6c-0131-43f0-a9ec-8db09eeb6e5d admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:28.060602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a053d6c-0131-43f0-a9ec-8db09eeb6e5d admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:28.113278 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc70ac-608c-469e-ac17-ab3b56d993c5 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:28.114979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-6dbc70ac-608c-469e-ac17-ab3b56d993c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:28.121075 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c72136-ee03-4ab5-8049-8d17b525776c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:28.121700 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c72136-ee03-4ab5-8049-8d17b525776c admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:28.121985 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c72136-ee03-4ab5-8049-8d17b525776c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:28.122258 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c72136-ee03-4ab5-8049-8d17b525776c admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:28.122887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c72136-ee03-4ab5-8049-8d17b525776c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:28.124847 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c72136-ee03-4ab5-8049-8d17b525776c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:28.135235 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:28.149505 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 748/1485] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:28 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:28.153749 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c72136-ee03-4ab5-8049-8d17b525776c admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:28.163535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c72136-ee03-4ab5-8049-8d17b525776c admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:28.169064 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1922e-7724-41e9-acd2-1a1f622ce805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:28.169668 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1922e-7724-41e9-acd2-1a1f622ce805 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:28.169947 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1922e-7724-41e9-acd2-1a1f622ce805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:28.170218 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1922e-7724-41e9-acd2-1a1f622ce805 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:28.171051 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1922e-7724-41e9-acd2-1a1f622ce805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:28.174068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1922e-7724-41e9-acd2-1a1f622ce805 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:28.188160 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 749/1486] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:28.189206 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc70ac-608c-469e-ac17-ab3b56d993c5 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:28.190705 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1922e-7724-41e9-acd2-1a1f622ce805 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:28.199364 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdce3c7-bb5d-47d6-8cb0-72324a403be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:28.200313 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdce3c7-bb5d-47d6-8cb0-72324a403be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:28.200591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdce3c7-bb5d-47d6-8cb0-72324a403be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:28.200860 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdce3c7-bb5d-47d6-8cb0-72324a403be0 admin admin] PATH_INFO: `/v3/users/dd8459a660564c288f3ba82318d7318e` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:28.202118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1922e-7724-41e9-acd2-1a1f622ce805 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:28.202393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdce3c7-bb5d-47d6-8cb0-72324a403be0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd8459a660564c288f3ba82318d7318e)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:28.204884 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdce3c7-bb5d-47d6-8cb0-72324a403be0 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:28.208462 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167b58c-2562-4f67-b1eb-465ef164541b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:28.209377 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167b58c-2562-4f67-b1eb-465ef164541b admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:28.209661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167b58c-2562-4f67-b1eb-465ef164541b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:28.209945 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167b58c-2562-4f67-b1eb-465ef164541b admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:28.210578 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167b58c-2562-4f67-b1eb-465ef164541b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:28.212324 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167b58c-2562-4f67-b1eb-465ef164541b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:28.225365 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 750/1487] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:28.229240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167b58c-2562-4f67-b1eb-465ef164541b admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:28.240650 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167b58c-2562-4f67-b1eb-465ef164541b admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:28.245823 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c87f9-937f-48aa-ae6b-9b8333f9f661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:28.246488 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c87f9-937f-48aa-ae6b-9b8333f9f661 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:28.246785 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c87f9-937f-48aa-ae6b-9b8333f9f661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:28.247017 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c87f9-937f-48aa-ae6b-9b8333f9f661 admin admin] PATH_INFO: `/v3/projects/f9216973754e4cfeacbd152b8f4822d9/users/8dba8d1d2e6442008c7ed47a214828cf/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:28.262211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c87f9-937f-48aa-ae6b-9b8333f9f661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9216973754e4cfeacbd152b8f4822d9, user_id=8dba8d1d2e6442008c7ed47a214828cf, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:28.264799 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c87f9-937f-48aa-ae6b-9b8333f9f661 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:28.268838 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bcdce3c7-bb5d-47d6-8cb0-72324a403be0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd8459a660564c288f3ba82318d7318e'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:28.269567 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bcdce3c7-bb5d-47d6-8cb0-72324a403be0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd8459a660564c288f3ba82318d7318e'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:28.274181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:28.279478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 751/1488] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:28 2022] PUT /identity/v3/projects/f9216973754e4cfeacbd152b8f4822d9/users/8dba8d1d2e6442008c7ed47a214828cf/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:28.283370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bcdce3c7-bb5d-47d6-8cb0-72324a403be0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd8459a660564c288f3ba82318d7318e'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:28.291153 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee1d7f-d496-4b87-8113-6f9987ec0cc6 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:28.291312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bcdce3c7-bb5d-47d6-8cb0-72324a403be0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd8459a660564c288f3ba82318d7318e'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:28.291412 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee1d7f-d496-4b87-8113-6f9987ec0cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:28.291724 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee1d7f-d496-4b87-8113-6f9987ec0cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:28.291961 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bcdce3c7-bb5d-47d6-8cb0-72324a403be0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd8459a660564c288f3ba82318d7318e'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:28.305026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 738/1489] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:28 2022] DELETE /identity/v3/users/dd8459a660564c288f3ba82318d7318e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:28.309422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd3118-ba52-4ac6-8798-e0e2f66c7873 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:28.309712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd3118-ba52-4ac6-8798-e0e2f66c7873 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:28.309985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd3118-ba52-4ac6-8798-e0e2f66c7873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:28.324497 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-9dee1d7f-d496-4b87-8113-6f9987ec0cc6 None None] MFA Rules not processed for user `8dba8d1d2e6442008c7ed47a214828cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:28.325695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9dee1d7f-d496-4b87-8113-6f9987ec0cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:28.329772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-e3cd3118-ba52-4ac6-8798-e0e2f66c7873 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:28.330949 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-e3cd3118-ba52-4ac6-8798-e0e2f66c7873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:28.389448 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 752/1490] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:28.393594 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee1d7f-d496-4b87-8113-6f9987ec0cc6 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:28.395525 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9dee1d7f-d496-4b87-8113-6f9987ec0cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:28.396196 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 739/1491] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:28.406251 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd3118-ba52-4ac6-8798-e0e2f66c7873 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:28.407611 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-e3cd3118-ba52-4ac6-8798-e0e2f66c7873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:28.411066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee1d7f-d496-4b87-8113-6f9987ec0cc6 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:28.415853 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305da3b-3a0f-461c-8db6-18fdd745f7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:28.416939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305da3b-3a0f-461c-8db6-18fdd745f7a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:28.416939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305da3b-3a0f-461c-8db6-18fdd745f7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:28.417112 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305da3b-3a0f-461c-8db6-18fdd745f7a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:28.417610 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305da3b-3a0f-461c-8db6-18fdd745f7a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:28.419735 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305da3b-3a0f-461c-8db6-18fdd745f7a7 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:28.451405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 753/1492] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:28 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:28.455331 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305da3b-3a0f-461c-8db6-18fdd745f7a7 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:28.464674 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:28.483949 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd3118-ba52-4ac6-8798-e0e2f66c7873 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:28.545916 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305da3b-3a0f-461c-8db6-18fdd745f7a7 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:28.551789 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c5abb-b1c6-45a2-b8e7-417cb31aa3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:28.552602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c5abb-b1c6-45a2-b8e7-417cb31aa3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:28.552943 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c5abb-b1c6-45a2-b8e7-417cb31aa3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:28.553283 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c5abb-b1c6-45a2-b8e7-417cb31aa3a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:28.554036 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c5abb-b1c6-45a2-b8e7-417cb31aa3a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:28.556094 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c5abb-b1c6-45a2-b8e7-417cb31aa3a8 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:28.561181 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd40c3-a720-4ae7-b6ec-0842160104ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:28.562041 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd40c3-a720-4ae7-b6ec-0842160104ee service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:28.562500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd40c3-a720-4ae7-b6ec-0842160104ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:28.562951 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd40c3-a720-4ae7-b6ec-0842160104ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:28.573513 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd40c3-a720-4ae7-b6ec-0842160104ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:28.575069 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd40c3-a720-4ae7-b6ec-0842160104ee service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:28.584152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 754/1493] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:28 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:21:28.586203 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 740/1494] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:21:28.588084 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c5abb-b1c6-45a2-b8e7-417cb31aa3a8 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:28.598804 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c5abb-b1c6-45a2-b8e7-417cb31aa3a8 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:28.603486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5351bfd-edbb-4d6d-ba99-d34d458965d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:28.604136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5351bfd-edbb-4d6d-ba99-d34d458965d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:28.604415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5351bfd-edbb-4d6d-ba99-d34d458965d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:28.604687 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5351bfd-edbb-4d6d-ba99-d34d458965d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:28.605318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5351bfd-edbb-4d6d-ba99-d34d458965d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:28.606924 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5351bfd-edbb-4d6d-ba99-d34d458965d5 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:28.617089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 755/1495] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:21:28.621345 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd40c3-a720-4ae7-b6ec-0842160104ee service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:28.632438 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd40c3-a720-4ae7-b6ec-0842160104ee service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:28.637106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af467cca-f838-4151-bf4f-686c1434d37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:28.637768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af467cca-f838-4151-bf4f-686c1434d37b admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:28.638134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af467cca-f838-4151-bf4f-686c1434d37b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:28.638418 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af467cca-f838-4151-bf4f-686c1434d37b admin admin] PATH_INFO: `/v3/projects/4b485ceb4bb84c4f9ed273944590d153/users/a659cc39dce24906b9239c5f45676cd0/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:28.654083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af467cca-f838-4151-bf4f-686c1434d37b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b485ceb4bb84c4f9ed273944590d153, user_id=a659cc39dce24906b9239c5f45676cd0, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:28.656802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af467cca-f838-4151-bf4f-686c1434d37b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:28.671163 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 741/1496] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:28 2022] PUT /identity/v3/projects/4b485ceb4bb84c4f9ed273944590d153/users/a659cc39dce24906b9239c5f45676cd0/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:28.674758 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5351bfd-edbb-4d6d-ba99-d34d458965d5 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:28.686836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5351bfd-edbb-4d6d-ba99-d34d458965d5 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:28.760120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:28.784909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cef125-daee-4616-87ef-0a65bf381a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:28.785610 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cef125-daee-4616-87ef-0a65bf381a16 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:28.785896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cef125-daee-4616-87ef-0a65bf381a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:28.786169 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cef125-daee-4616-87ef-0a65bf381a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:28.786838 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cef125-daee-4616-87ef-0a65bf381a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:28.788638 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cef125-daee-4616-87ef-0a65bf381a16 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:28.803834 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 756/1497] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:28.805399 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af467cca-f838-4151-bf4f-686c1434d37b admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:28.815869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af467cca-f838-4151-bf4f-686c1434d37b admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:28.822204 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49c4e7-c471-497a-bda8-0af94e41d1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:28.822885 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49c4e7-c471-497a-bda8-0af94e41d1e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:28.823166 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49c4e7-c471-497a-bda8-0af94e41d1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:28.823440 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49c4e7-c471-497a-bda8-0af94e41d1e7 admin admin] PATH_INFO: `/v3/projects/4b485ceb4bb84c4f9ed273944590d153/users/a659cc39dce24906b9239c5f45676cd0/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:28.825442 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49c4e7-c471-497a-bda8-0af94e41d1e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b485ceb4bb84c4f9ed273944590d153, user_id=a659cc39dce24906b9239c5f45676cd0, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:28.828124 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49c4e7-c471-497a-bda8-0af94e41d1e7 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:28.839759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 742/1498] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:28 2022] PUT /identity/v3/projects/4b485ceb4bb84c4f9ed273944590d153/users/a659cc39dce24906b9239c5f45676cd0/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:28.850467 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c45757-e84c-41f9-8bd5-7583c075f209 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:28.850988 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c45757-e84c-41f9-8bd5-7583c075f209 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:28.851227 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c45757-e84c-41f9-8bd5-7583c075f209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:28.878619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-66c45757-e84c-41f9-8bd5-7583c075f209 None None] MFA Rules not processed for user `a659cc39dce24906b9239c5f45676cd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:28.879930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-66c45757-e84c-41f9-8bd5-7583c075f209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:28.949527 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49c4e7-c471-497a-bda8-0af94e41d1e7 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:28.952946 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-6a49c4e7-c471-497a-bda8-0af94e41d1e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:28.953280 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 757/1499] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:28.965420 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c45757-e84c-41f9-8bd5-7583c075f209 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:28.967023 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-66c45757-e84c-41f9-8bd5-7583c075f209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:29.034709 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c45757-e84c-41f9-8bd5-7583c075f209 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:29.038565 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928ab40-2c14-4d06-b9b3-933583baefec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:29.039092 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928ab40-2c14-4d06-b9b3-933583baefec service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:29.040310 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928ab40-2c14-4d06-b9b3-933583baefec service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:29.040593 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928ab40-2c14-4d06-b9b3-933583baefec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:29.044889 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49c4e7-c471-497a-bda8-0af94e41d1e7 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:29.050727 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c0009-50de-41e3-bccf-1722d52433c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:29.051366 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c0009-50de-41e3-bccf-1722d52433c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:29.051772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c0009-50de-41e3-bccf-1722d52433c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:29.052059 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c0009-50de-41e3-bccf-1722d52433c0 admin admin] PATH_INFO: `/v3/projects/086967943cb94332b5e7eee01e997c73` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:29.053020 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928ab40-2c14-4d06-b9b3-933583baefec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:29.053193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c0009-50de-41e3-bccf-1722d52433c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=086967943cb94332b5e7eee01e997c73)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:29.054500 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928ab40-2c14-4d06-b9b3-933583baefec service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:29.055279 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c0009-50de-41e3-bccf-1722d52433c0 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:29.070025 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 758/1500] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:21:29.113356 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-968c0009-50de-41e3-bccf-1722d52433c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '086967943cb94332b5e7eee01e997c73', 'request_id': 'req-968c0009-50de-41e3-bccf-1722d52433c0'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.122652 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-968c0009-50de-41e3-bccf-1722d52433c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '086967943cb94332b5e7eee01e997c73', 'request_id': 'req-968c0009-50de-41e3-bccf-1722d52433c0'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.127512 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:29.130370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-968c0009-50de-41e3-bccf-1722d52433c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '086967943cb94332b5e7eee01e997c73', 'request_id': 'req-968c0009-50de-41e3-bccf-1722d52433c0'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.145998 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-968c0009-50de-41e3-bccf-1722d52433c0 admin admin] The token cache is being invalidate because project 086967943cb94332b5e7eee01e997c73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:29.146295 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-968c0009-50de-41e3-bccf-1722d52433c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.147242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-968c0009-50de-41e3-bccf-1722d52433c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.149207 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 743/1501] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:28 2022] DELETE /identity/v3/projects/086967943cb94332b5e7eee01e997c73 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:29.152895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928ab40-2c14-4d06-b9b3-933583baefec service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:29.163917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928ab40-2c14-4d06-b9b3-933583baefec service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:29.245588 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846378c-2f1c-4fd1-b56d-1c871fc49cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:29.246241 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846378c-2f1c-4fd1-b56d-1c871fc49cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:29.246637 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846378c-2f1c-4fd1-b56d-1c871fc49cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:29.246772 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846378c-2f1c-4fd1-b56d-1c871fc49cb2 admin admin] PATH_INFO: `/v3/users/117bf14d7fba4430ae2f058ca3c29667` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:29.248300 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f846378c-2f1c-4fd1-b56d-1c871fc49cb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=117bf14d7fba4430ae2f058ca3c29667)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:29.250072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f846378c-2f1c-4fd1-b56d-1c871fc49cb2 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:29.266421 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:29.314400 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-f846378c-2f1c-4fd1-b56d-1c871fc49cb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '117bf14d7fba4430ae2f058ca3c29667'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.334134 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-f846378c-2f1c-4fd1-b56d-1c871fc49cb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '117bf14d7fba4430ae2f058ca3c29667'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.342786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-f846378c-2f1c-4fd1-b56d-1c871fc49cb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '117bf14d7fba4430ae2f058ca3c29667'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.343503 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-f846378c-2f1c-4fd1-b56d-1c871fc49cb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '117bf14d7fba4430ae2f058ca3c29667'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.353307 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-f846378c-2f1c-4fd1-b56d-1c871fc49cb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '117bf14d7fba4430ae2f058ca3c29667'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.361211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 759/1502] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:29 2022] DELETE /identity/v3/users/117bf14d7fba4430ae2f058ca3c29667 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:29.562384 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c0009-50de-41e3-bccf-1722d52433c0 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:29.563945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-968c0009-50de-41e3-bccf-1722d52433c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:29.578659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c0009-50de-41e3-bccf-1722d52433c0 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:29.669351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a7fdf-6fce-4dbb-b628-bc55258999ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:29.669989 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a7fdf-6fce-4dbb-b628-bc55258999ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:29.670266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a7fdf-6fce-4dbb-b628-bc55258999ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:29.671210 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a7fdf-6fce-4dbb-b628-bc55258999ce admin admin] PATH_INFO: `/v3/users/8dba8d1d2e6442008c7ed47a214828cf` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:29.671683 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a7fdf-6fce-4dbb-b628-bc55258999ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dba8d1d2e6442008c7ed47a214828cf)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:29.673583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a7fdf-6fce-4dbb-b628-bc55258999ce admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:29.725405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846378c-2f1c-4fd1-b56d-1c871fc49cb2 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:29.726933 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f846378c-2f1c-4fd1-b56d-1c871fc49cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:29.738065 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-465a7fdf-6fce-4dbb-b628-bc55258999ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dba8d1d2e6442008c7ed47a214828cf'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.738730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-465a7fdf-6fce-4dbb-b628-bc55258999ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dba8d1d2e6442008c7ed47a214828cf'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.748297 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846378c-2f1c-4fd1-b56d-1c871fc49cb2 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:29.752181 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-465a7fdf-6fce-4dbb-b628-bc55258999ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dba8d1d2e6442008c7ed47a214828cf'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.761540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-465a7fdf-6fce-4dbb-b628-bc55258999ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dba8d1d2e6442008c7ed47a214828cf'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.762333 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-465a7fdf-6fce-4dbb-b628-bc55258999ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dba8d1d2e6442008c7ed47a214828cf'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.763971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ed173-ee05-42a5-827f-b8b4a2293033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:29.764726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ed173-ee05-42a5-827f-b8b4a2293033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:29.765010 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ed173-ee05-42a5-827f-b8b4a2293033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:29.765357 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ed173-ee05-42a5-827f-b8b4a2293033 admin admin] PATH_INFO: `/v3/projects/f222279eb00a489da47816ea31c0edf8` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:29.766648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ed173-ee05-42a5-827f-b8b4a2293033 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f222279eb00a489da47816ea31c0edf8)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:29.768656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ed173-ee05-42a5-827f-b8b4a2293033 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:29.777254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 744/1503] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:29 2022] DELETE /identity/v3/users/8dba8d1d2e6442008c7ed47a214828cf => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:29.781748 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae50c67-8a1f-403a-a663-37fa99a35f6c None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:29.782997 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae50c67-8a1f-403a-a663-37fa99a35f6c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:29.783271 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae50c67-8a1f-403a-a663-37fa99a35f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:29.810028 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-aae50c67-8a1f-403a-a663-37fa99a35f6c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:29.811260 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-aae50c67-8a1f-403a-a663-37fa99a35f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:29.838237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2e0ed173-ee05-42a5-827f-b8b4a2293033 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f222279eb00a489da47816ea31c0edf8', 'request_id': 'req-2e0ed173-ee05-42a5-827f-b8b4a2293033'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.847342 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2e0ed173-ee05-42a5-827f-b8b4a2293033 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f222279eb00a489da47816ea31c0edf8', 'request_id': 'req-2e0ed173-ee05-42a5-827f-b8b4a2293033'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.856132 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2e0ed173-ee05-42a5-827f-b8b4a2293033 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f222279eb00a489da47816ea31c0edf8', 'request_id': 'req-2e0ed173-ee05-42a5-827f-b8b4a2293033'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.871690 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2e0ed173-ee05-42a5-827f-b8b4a2293033 admin admin] The token cache is being invalidate because project f222279eb00a489da47816ea31c0edf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:29.871978 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2e0ed173-ee05-42a5-827f-b8b4a2293033 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.872607 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2e0ed173-ee05-42a5-827f-b8b4a2293033 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:29.874337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 760/1504] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:29 2022] DELETE /identity/v3/projects/f222279eb00a489da47816ea31c0edf8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:29.876897 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 745/1505] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:29.880604 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3af6be-68d5-4d2d-8823-2dec3d19535b None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:29.880839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3af6be-68d5-4d2d-8823-2dec3d19535b None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:29.881126 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3af6be-68d5-4d2d-8823-2dec3d19535b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:29.888539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae50c67-8a1f-403a-a663-37fa99a35f6c None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:29.889963 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-aae50c67-8a1f-403a-a663-37fa99a35f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:29.902425 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:29.903635 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-6a3af6be-68d5-4d2d-8823-2dec3d19535b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:29.904633 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-6a3af6be-68d5-4d2d-8823-2dec3d19535b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:29.963250 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae50c67-8a1f-403a-a663-37fa99a35f6c None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:29.968499 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 761/1506] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:29.976685 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee878ff3-9058-4af4-8f7d-c00196349e2d None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:29.976976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee878ff3-9058-4af4-8f7d-c00196349e2d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:29.977256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee878ff3-9058-4af4-8f7d-c00196349e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:29.995994 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-ee878ff3-9058-4af4-8f7d-c00196349e2d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:29.997181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ee878ff3-9058-4af4-8f7d-c00196349e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:30.005489 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 762/1507] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:30.009671 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee878ff3-9058-4af4-8f7d-c00196349e2d None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:30.011537 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88268c7d-b5cd-49e0-88f3-8473efdffac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:30.012184 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88268c7d-b5cd-49e0-88f3-8473efdffac4 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:30.012516 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88268c7d-b5cd-49e0-88f3-8473efdffac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:30.012727 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88268c7d-b5cd-49e0-88f3-8473efdffac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:30.014758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-88268c7d-b5cd-49e0-88f3-8473efdffac4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:30.019527 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee878ff3-9058-4af4-8f7d-c00196349e2d None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:30.024375 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f9b59-49bd-423c-94bb-c7322bdd5158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:30.025077 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f9b59-49bd-423c-94bb-c7322bdd5158 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:30.025400 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f9b59-49bd-423c-94bb-c7322bdd5158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:30.025685 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f9b59-49bd-423c-94bb-c7322bdd5158 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:30.026353 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f9b59-49bd-423c-94bb-c7322bdd5158 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:21:30.026681 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f9b59-49bd-423c-94bb-c7322bdd5158 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:30.027832 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88268c7d-b5cd-49e0-88f3-8473efdffac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:30.028325 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f9b59-49bd-423c-94bb-c7322bdd5158 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:30.029254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88268c7d-b5cd-49e0-88f3-8473efdffac4 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:30.029823 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:30.039005 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 746/1508] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:21:30.044804 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 763/1509] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:21:30.045781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88268c7d-b5cd-49e0-88f3-8473efdffac4 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:30.055138 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88268c7d-b5cd-49e0-88f3-8473efdffac4 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:30.059797 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6fce6-a8a1-4093-93c7-37662ab416d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:30.060392 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6fce6-a8a1-4093-93c7-37662ab416d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:30.060674 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6fce6-a8a1-4093-93c7-37662ab416d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:30.060947 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6fce6-a8a1-4093-93c7-37662ab416d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:30.061571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6fce6-a8a1-4093-93c7-37662ab416d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:30.063479 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6fce6-a8a1-4093-93c7-37662ab416d4 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:30.094869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 747/1510] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:30 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:21:30.096145 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f9b59-49bd-423c-94bb-c7322bdd5158 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:30.108122 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f9b59-49bd-423c-94bb-c7322bdd5158 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:30.169056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d7d56-c7a5-4e95-88ef-58eb7427c56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:30.169677 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d7d56-c7a5-4e95-88ef-58eb7427c56e admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:30.170020 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d7d56-c7a5-4e95-88ef-58eb7427c56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:30.170214 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d7d56-c7a5-4e95-88ef-58eb7427c56e admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:30.170841 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d7d56-c7a5-4e95-88ef-58eb7427c56e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:30.172868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d7d56-c7a5-4e95-88ef-58eb7427c56e admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:30.215851 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 764/1511] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:30 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:21:30.217028 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6fce6-a8a1-4093-93c7-37662ab416d4 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:30.229723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6fce6-a8a1-4093-93c7-37662ab416d4 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:30.234747 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b18815-073a-4cca-9e53-2a4211cf7513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:30.235539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b18815-073a-4cca-9e53-2a4211cf7513 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:30.235859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b18815-073a-4cca-9e53-2a4211cf7513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:30.236134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b18815-073a-4cca-9e53-2a4211cf7513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:30.236774 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b18815-073a-4cca-9e53-2a4211cf7513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:30.238962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b18815-073a-4cca-9e53-2a4211cf7513 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:30.250553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 748/1512] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:21:30.254677 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d7d56-c7a5-4e95-88ef-58eb7427c56e admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:30.265312 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d7d56-c7a5-4e95-88ef-58eb7427c56e admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:30.270918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f6e96-fc8c-4108-95a4-6b4566876888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:30.271830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f6e96-fc8c-4108-95a4-6b4566876888 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:30.272185 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f6e96-fc8c-4108-95a4-6b4566876888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:30.273566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f6e96-fc8c-4108-95a4-6b4566876888 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:30.273927 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f6e96-fc8c-4108-95a4-6b4566876888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:30.275808 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f6e96-fc8c-4108-95a4-6b4566876888 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:30.289341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 765/1513] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:30.293492 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b18815-073a-4cca-9e53-2a4211cf7513 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:30.305118 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b18815-073a-4cca-9e53-2a4211cf7513 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:30.309787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e6c13-362c-4e49-b383-b3d890671062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:30.310525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e6c13-362c-4e49-b383-b3d890671062 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:30.310878 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e6c13-362c-4e49-b383-b3d890671062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:30.311221 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e6c13-362c-4e49-b383-b3d890671062 admin admin] PATH_INFO: `/v3/projects/06c62ce86b224a0c9249b0b7f5477979/users/cfce2b8a66354c9888279fdd8eca11d6/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:30.328271 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e6c13-362c-4e49-b383-b3d890671062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06c62ce86b224a0c9249b0b7f5477979, user_id=cfce2b8a66354c9888279fdd8eca11d6, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:30.330962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e6c13-362c-4e49-b383-b3d890671062 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:30.347791 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f6e96-fc8c-4108-95a4-6b4566876888 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:30.348971 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 749/1514] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:30 2022] PUT /identity/v3/projects/06c62ce86b224a0c9249b0b7f5477979/users/cfce2b8a66354c9888279fdd8eca11d6/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:30.357976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-140f6e96-fc8c-4108-95a4-6b4566876888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:30.361909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60533d4c-adf6-454f-81f6-0316a597d351 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:30.362197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60533d4c-adf6-454f-81f6-0316a597d351 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:30.363227 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60533d4c-adf6-454f-81f6-0316a597d351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:30.373815 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f6e96-fc8c-4108-95a4-6b4566876888 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:30.393559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-60533d4c-adf6-454f-81f6-0316a597d351 None None] MFA Rules not processed for user `cfce2b8a66354c9888279fdd8eca11d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:30.395225 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-60533d4c-adf6-454f-81f6-0316a597d351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:30.443189 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95192a-9d72-4509-8033-cd287577b760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:30.443893 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95192a-9d72-4509-8033-cd287577b760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:30.444170 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95192a-9d72-4509-8033-cd287577b760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:30.444441 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95192a-9d72-4509-8033-cd287577b760 admin admin] PATH_INFO: `/v3/projects/f9216973754e4cfeacbd152b8f4822d9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:30.445584 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95192a-9d72-4509-8033-cd287577b760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9216973754e4cfeacbd152b8f4822d9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:30.447489 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95192a-9d72-4509-8033-cd287577b760 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:30.471560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:30.478728 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 750/1515] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:30.483160 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60533d4c-adf6-454f-81f6-0316a597d351 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:30.522190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9c95192a-9d72-4509-8033-cd287577b760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9216973754e4cfeacbd152b8f4822d9', 'request_id': 'req-9c95192a-9d72-4509-8033-cd287577b760'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:30.527038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60533d4c-adf6-454f-81f6-0316a597d351 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:30.548844 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9c95192a-9d72-4509-8033-cd287577b760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9216973754e4cfeacbd152b8f4822d9', 'request_id': 'req-9c95192a-9d72-4509-8033-cd287577b760'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:30.558757 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9c95192a-9d72-4509-8033-cd287577b760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9216973754e4cfeacbd152b8f4822d9', 'request_id': 'req-9c95192a-9d72-4509-8033-cd287577b760'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:30.574454 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9c95192a-9d72-4509-8033-cd287577b760 admin admin] The token cache is being invalidate because project f9216973754e4cfeacbd152b8f4822d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:30.574976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9c95192a-9d72-4509-8033-cd287577b760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:30.575881 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9c95192a-9d72-4509-8033-cd287577b760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:30.580456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 766/1516] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:30 2022] DELETE /identity/v3/projects/f9216973754e4cfeacbd152b8f4822d9 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:30.584729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95192a-9d72-4509-8033-cd287577b760 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:30.586417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9c95192a-9d72-4509-8033-cd287577b760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:30.592472 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:30.609876 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c264836-8d02-4a25-9526-5a8e6c6ef06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:30.610852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c264836-8d02-4a25-9526-5a8e6c6ef06b admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:30.611303 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c264836-8d02-4a25-9526-5a8e6c6ef06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:30.611941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c264836-8d02-4a25-9526-5a8e6c6ef06b admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:30.613090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c264836-8d02-4a25-9526-5a8e6c6ef06b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:30.615145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c264836-8d02-4a25-9526-5a8e6c6ef06b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:30.648924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 751/1517] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:30 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:30.653078 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c264836-8d02-4a25-9526-5a8e6c6ef06b admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:30.654758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3c264836-8d02-4a25-9526-5a8e6c6ef06b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:30.668176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95192a-9d72-4509-8033-cd287577b760 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:30.724505 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c264836-8d02-4a25-9526-5a8e6c6ef06b admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:30.739386 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777631f-da31-4823-b63a-ebaf0dd14db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:30.740239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777631f-da31-4823-b63a-ebaf0dd14db2 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:30.740664 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777631f-da31-4823-b63a-ebaf0dd14db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:30.741212 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777631f-da31-4823-b63a-ebaf0dd14db2 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:30.741853 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777631f-da31-4823-b63a-ebaf0dd14db2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:30.743841 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777631f-da31-4823-b63a-ebaf0dd14db2 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:30.760286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d8141-5649-4a5a-9836-070c44e9c234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:30.761498 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d8141-5649-4a5a-9836-070c44e9c234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:30.762194 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d8141-5649-4a5a-9836-070c44e9c234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:30.762608 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d8141-5649-4a5a-9836-070c44e9c234 admin admin] PATH_INFO: `/v3/users/a659cc39dce24906b9239c5f45676cd0` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:30.764248 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d8141-5649-4a5a-9836-070c44e9c234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a659cc39dce24906b9239c5f45676cd0)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:30.766368 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d8141-5649-4a5a-9836-070c44e9c234 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:30.767881 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 752/1518] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:30 2022] POST /identity/v3/users => generated 298 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:30.772698 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777631f-da31-4823-b63a-ebaf0dd14db2 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:30.783952 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777631f-da31-4823-b63a-ebaf0dd14db2 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:30.788736 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67de13-6224-4eb9-95cd-d2440b6b709e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:30.789414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67de13-6224-4eb9-95cd-d2440b6b709e admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:30.789777 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67de13-6224-4eb9-95cd-d2440b6b709e admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:30.790115 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67de13-6224-4eb9-95cd-d2440b6b709e admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:30.790877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67de13-6224-4eb9-95cd-d2440b6b709e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:30.792565 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67de13-6224-4eb9-95cd-d2440b6b709e admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:30.805941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 753/1519] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:21:30.811714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67de13-6224-4eb9-95cd-d2440b6b709e admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:30.823998 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67de13-6224-4eb9-95cd-d2440b6b709e admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:30.835937 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-120d8141-5649-4a5a-9836-070c44e9c234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a659cc39dce24906b9239c5f45676cd0'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:30.837875 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ea266-fb12-4a96-82f3-a4d9f1edcc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:30.839141 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ea266-fb12-4a96-82f3-a4d9f1edcc40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:30.839983 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ea266-fb12-4a96-82f3-a4d9f1edcc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:30.840405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ea266-fb12-4a96-82f3-a4d9f1edcc40 admin admin] PATH_INFO: `/v3/projects/4481acf2875f49e7a508d223fa6714a3/users/0d9429fda39545a09923752ba279a02c/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:30.850137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-120d8141-5649-4a5a-9836-070c44e9c234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a659cc39dce24906b9239c5f45676cd0'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:30.857531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ea266-fb12-4a96-82f3-a4d9f1edcc40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4481acf2875f49e7a508d223fa6714a3, user_id=0d9429fda39545a09923752ba279a02c, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:30.858048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-120d8141-5649-4a5a-9836-070c44e9c234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a659cc39dce24906b9239c5f45676cd0'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:30.858982 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-120d8141-5649-4a5a-9836-070c44e9c234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a659cc39dce24906b9239c5f45676cd0'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:30.860393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ea266-fb12-4a96-82f3-a4d9f1edcc40 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:30.869251 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-120d8141-5649-4a5a-9836-070c44e9c234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a659cc39dce24906b9239c5f45676cd0'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:30.874031 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 754/1520] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:30 2022] PUT /identity/v3/projects/4481acf2875f49e7a508d223fa6714a3/users/0d9429fda39545a09923752ba279a02c/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:30.875804 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 767/1521] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:30 2022] DELETE /identity/v3/users/a659cc39dce24906b9239c5f45676cd0 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:30.877897 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ea266-fb12-4a96-82f3-a4d9f1edcc40 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:30.880291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-0b0ea266-fb12-4a96-82f3-a4d9f1edcc40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:30.924631 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:30.947763 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ea266-fb12-4a96-82f3-a4d9f1edcc40 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:31.011306 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b676851-7824-481d-9ed7-61dc0a89f2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:31.011955 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b676851-7824-481d-9ed7-61dc0a89f2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:31.012244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b676851-7824-481d-9ed7-61dc0a89f2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:31.012519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b676851-7824-481d-9ed7-61dc0a89f2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:31.013157 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b676851-7824-481d-9ed7-61dc0a89f2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:31.014788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b676851-7824-481d-9ed7-61dc0a89f2d4 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:31.025252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 755/1522] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:21:31.029312 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d8141-5649-4a5a-9836-070c44e9c234 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:31.039889 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d8141-5649-4a5a-9836-070c44e9c234 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:31.044417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2929012-0b46-4dd2-951c-7349e1e6005b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:31.045082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2929012-0b46-4dd2-951c-7349e1e6005b admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:31.045364 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2929012-0b46-4dd2-951c-7349e1e6005b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:31.045633 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2929012-0b46-4dd2-951c-7349e1e6005b admin admin] PATH_INFO: `/v3/projects/4481acf2875f49e7a508d223fa6714a3/users/0d9429fda39545a09923752ba279a02c/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:31.047532 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2929012-0b46-4dd2-951c-7349e1e6005b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4481acf2875f49e7a508d223fa6714a3, user_id=0d9429fda39545a09923752ba279a02c, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:31.050789 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2929012-0b46-4dd2-951c-7349e1e6005b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:31.062093 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 768/1523] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:31 2022] PUT /identity/v3/projects/4481acf2875f49e7a508d223fa6714a3/users/0d9429fda39545a09923752ba279a02c/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:31.073381 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a38ef-223f-4cdd-a57c-027f233d83a1 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:31.074498 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a38ef-223f-4cdd-a57c-027f233d83a1 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:31.075052 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a38ef-223f-4cdd-a57c-027f233d83a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:31.110236 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-8a9a38ef-223f-4cdd-a57c-027f233d83a1 None None] MFA Rules not processed for user `0d9429fda39545a09923752ba279a02c`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:31.111464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8a9a38ef-223f-4cdd-a57c-027f233d83a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:31.188571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 756/1524] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:31 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:31.201424 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2929012-0b46-4dd2-951c-7349e1e6005b admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:31.206385 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d2929012-0b46-4dd2-951c-7349e1e6005b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:31.207021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a38ef-223f-4cdd-a57c-027f233d83a1 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:31.208309 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8a9a38ef-223f-4cdd-a57c-027f233d83a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:31.231334 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:31.285360 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2929012-0b46-4dd2-951c-7349e1e6005b admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:31.288881 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a38ef-223f-4cdd-a57c-027f233d83a1 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:31.289408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac3b03-e053-40db-9420-9cd53c2b37be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:31.290119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac3b03-e053-40db-9420-9cd53c2b37be service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:31.291001 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac3b03-e053-40db-9420-9cd53c2b37be service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:31.291001 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac3b03-e053-40db-9420-9cd53c2b37be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:31.294388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b5bc3-4a67-4026-9917-912f2a285e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:31.295025 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b5bc3-4a67-4026-9917-912f2a285e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:31.295306 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b5bc3-4a67-4026-9917-912f2a285e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:31.295606 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b5bc3-4a67-4026-9917-912f2a285e30 admin admin] PATH_INFO: `/v3/projects/4b485ceb4bb84c4f9ed273944590d153` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:31.296728 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b5bc3-4a67-4026-9917-912f2a285e30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b485ceb4bb84c4f9ed273944590d153)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:31.298639 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b5bc3-4a67-4026-9917-912f2a285e30 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:31.302915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac3b03-e053-40db-9420-9cd53c2b37be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:31.304392 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac3b03-e053-40db-9420-9cd53c2b37be service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:31.310331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:31.320307 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 769/1525] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:31.357448 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-2d6b5bc3-4a67-4026-9917-912f2a285e30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b485ceb4bb84c4f9ed273944590d153', 'request_id': 'req-2d6b5bc3-4a67-4026-9917-912f2a285e30'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:31.368755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-2d6b5bc3-4a67-4026-9917-912f2a285e30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b485ceb4bb84c4f9ed273944590d153', 'request_id': 'req-2d6b5bc3-4a67-4026-9917-912f2a285e30'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:31.377211 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-2d6b5bc3-4a67-4026-9917-912f2a285e30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b485ceb4bb84c4f9ed273944590d153', 'request_id': 'req-2d6b5bc3-4a67-4026-9917-912f2a285e30'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:31.389691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-2d6b5bc3-4a67-4026-9917-912f2a285e30 admin admin] The token cache is being invalidate because project 4b485ceb4bb84c4f9ed273944590d153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:31.390564 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-2d6b5bc3-4a67-4026-9917-912f2a285e30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:31.391261 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-2d6b5bc3-4a67-4026-9917-912f2a285e30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:31.393344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 757/1526] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:31 2022] DELETE /identity/v3/projects/4b485ceb4bb84c4f9ed273944590d153 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:31.398846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c2899-b35b-423f-a19b-fede3cfe076f None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:31.399159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c2899-b35b-423f-a19b-fede3cfe076f None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:31.399433 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c2899-b35b-423f-a19b-fede3cfe076f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:31.420119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-116c2899-b35b-423f-a19b-fede3cfe076f None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:31.421337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-116c2899-b35b-423f-a19b-fede3cfe076f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:31.446131 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b5bc3-4a67-4026-9917-912f2a285e30 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:31.448298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2d6b5bc3-4a67-4026-9917-912f2a285e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:31.497959 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 770/1527] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:31.501149 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a370de-7918-4a8d-9b3f-d11645cb2284 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:31.502918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a370de-7918-4a8d-9b3f-d11645cb2284 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:31.503613 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a370de-7918-4a8d-9b3f-d11645cb2284 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:31.511475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 771/1528] 10.222.0.13 () {54 vars in 921 bytes} [Fri Oct 21 21:21:31 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:21:31.514691 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed01cfb-fee6-4594-830e-26b652d57960 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:31.514976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed01cfb-fee6-4594-830e-26b652d57960 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:31.515286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed01cfb-fee6-4594-830e-26b652d57960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:31.551761 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-7ed01cfb-fee6-4594-830e-26b652d57960 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:31.552963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-7ed01cfb-fee6-4594-830e-26b652d57960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:31.553563 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b5bc3-4a67-4026-9917-912f2a285e30 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:31.567495 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9805fd5-9ed0-4d7e-ac31-c4089f447f36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:31.568151 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9805fd5-9ed0-4d7e-ac31-c4089f447f36 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:31.568433 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9805fd5-9ed0-4d7e-ac31-c4089f447f36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:31.568711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9805fd5-9ed0-4d7e-ac31-c4089f447f36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:31.570707 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d9805fd5-9ed0-4d7e-ac31-c4089f447f36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:31.579472 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 772/1529] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:31.580765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d828d8a-94be-4258-859b-885e2843fbe1 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:31.581056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d828d8a-94be-4258-859b-885e2843fbe1 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:31.581450 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d828d8a-94be-4258-859b-885e2843fbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:31.602385 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-2d828d8a-94be-4258-859b-885e2843fbe1 None None] MFA Rules not processed for user `12dad9f016a740b98538722e1b723823`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:31.604101 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2d828d8a-94be-4258-859b-885e2843fbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:31.647945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9805fd5-9ed0-4d7e-ac31-c4089f447f36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:31.649487 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9805fd5-9ed0-4d7e-ac31-c4089f447f36 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:31.658733 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 758/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:21:31.659380 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9805fd5-9ed0-4d7e-ac31-c4089f447f36 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:31.669569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9805fd5-9ed0-4d7e-ac31-c4089f447f36 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:31.674381 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7c9c4-78a8-4b16-9d41-0cbe4cfdc6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:31.674687 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 773/1531] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Oct 21 21:21:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:31.675009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7c9c4-78a8-4b16-9d41-0cbe4cfdc6c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:31.675289 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7c9c4-78a8-4b16-9d41-0cbe4cfdc6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:31.675558 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7c9c4-78a8-4b16-9d41-0cbe4cfdc6c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:31.676229 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7c9c4-78a8-4b16-9d41-0cbe4cfdc6c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:21:31.676523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7c9c4-78a8-4b16-9d41-0cbe4cfdc6c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:31.678232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7c9c4-78a8-4b16-9d41-0cbe4cfdc6c9 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:31.691215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d828d8a-94be-4258-859b-885e2843fbe1 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:31.691932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2d828d8a-94be-4258-859b-885e2843fbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:31.694801 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 759/1532] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:21:31.699323 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7c9c4-78a8-4b16-9d41-0cbe4cfdc6c9 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:31.705986 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d828d8a-94be-4258-859b-885e2843fbe1 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:31.712648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318ff98-7edb-41dd-a7be-c0aa1b8864c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:31.713248 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318ff98-7edb-41dd-a7be-c0aa1b8864c9 service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:31.713482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7c9c4-78a8-4b16-9d41-0cbe4cfdc6c9 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:31.713646 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318ff98-7edb-41dd-a7be-c0aa1b8864c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:31.713905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318ff98-7edb-41dd-a7be-c0aa1b8864c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:31.718904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e09302-d6b2-4c37-82f1-252b3872f218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:31.719134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e09302-d6b2-4c37-82f1-252b3872f218 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:31.719370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e09302-d6b2-4c37-82f1-252b3872f218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:31.719672 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e09302-d6b2-4c37-82f1-252b3872f218 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:31.720307 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e09302-d6b2-4c37-82f1-252b3872f218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:31.722837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e09302-d6b2-4c37-82f1-252b3872f218 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:31.726363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c318ff98-7edb-41dd-a7be-c0aa1b8864c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:31.730630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c318ff98-7edb-41dd-a7be-c0aa1b8864c9 service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:31.748519 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 774/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:21:31.759478 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 760/1534] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:31 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:21:31.763458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318ff98-7edb-41dd-a7be-c0aa1b8864c9 service nova] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:31.775158 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318ff98-7edb-41dd-a7be-c0aa1b8864c9 service nova] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:31.811137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:31.861724 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ac7b3-4dcc-4125-8a47-fe57bfbe5295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:31.863405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ac7b3-4dcc-4125-8a47-fe57bfbe5295 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:31.863940 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ac7b3-4dcc-4125-8a47-fe57bfbe5295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:31.865892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ac7b3-4dcc-4125-8a47-fe57bfbe5295 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:31.866676 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ac7b3-4dcc-4125-8a47-fe57bfbe5295 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:31.868768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ac7b3-4dcc-4125-8a47-fe57bfbe5295 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:31.924034 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 775/1535] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:31 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:21:31.929175 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e09302-d6b2-4c37-82f1-252b3872f218 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:31.943459 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e09302-d6b2-4c37-82f1-252b3872f218 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:31.951129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd7833-9db4-4025-8bd0-8fa67829404e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:31.951882 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd7833-9db4-4025-8bd0-8fa67829404e admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:31.952169 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd7833-9db4-4025-8bd0-8fa67829404e admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:31.952548 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd7833-9db4-4025-8bd0-8fa67829404e admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:31.953120 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd7833-9db4-4025-8bd0-8fa67829404e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:31.954858 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd7833-9db4-4025-8bd0-8fa67829404e admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:31.966778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 761/1536] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:31.975695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ac7b3-4dcc-4125-8a47-fe57bfbe5295 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:31.990097 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ac7b3-4dcc-4125-8a47-fe57bfbe5295 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:31.995160 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99179c5a-b077-4829-96a8-6cc14e85bd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:31.995845 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99179c5a-b077-4829-96a8-6cc14e85bd7c admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:31.996126 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99179c5a-b077-4829-96a8-6cc14e85bd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:31.996425 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99179c5a-b077-4829-96a8-6cc14e85bd7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:31.997062 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99179c5a-b077-4829-96a8-6cc14e85bd7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:31.998909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99179c5a-b077-4829-96a8-6cc14e85bd7c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:32.011533 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 776/1537] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:32.015094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd7833-9db4-4025-8bd0-8fa67829404e admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:32.028341 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd7833-9db4-4025-8bd0-8fa67829404e admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:32.033262 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180c61f-084e-4eb6-8049-15ff97e51f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:32.033936 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180c61f-084e-4eb6-8049-15ff97e51f98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:32.034293 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180c61f-084e-4eb6-8049-15ff97e51f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:32.034598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180c61f-084e-4eb6-8049-15ff97e51f98 admin admin] PATH_INFO: `/v3/projects/7ffd679007cd4fc8ab55ddf9cf6ca94f/users/1f2f0682f3d34de2b9ae5f4371550ff4/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:32.038571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:32.045227 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99179c5a-b077-4829-96a8-6cc14e85bd7c admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:32.047096 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-99179c5a-b077-4829-96a8-6cc14e85bd7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:32.055474 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180c61f-084e-4eb6-8049-15ff97e51f98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ffd679007cd4fc8ab55ddf9cf6ca94f, user_id=1f2f0682f3d34de2b9ae5f4371550ff4, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:32.058975 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180c61f-084e-4eb6-8049-15ff97e51f98 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:32.062509 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99179c5a-b077-4829-96a8-6cc14e85bd7c admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:32.068111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33bc8e-8dbd-4e44-8823-456c8b46dc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:32.068835 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33bc8e-8dbd-4e44-8823-456c8b46dc67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:32.069191 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33bc8e-8dbd-4e44-8823-456c8b46dc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:32.069581 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33bc8e-8dbd-4e44-8823-456c8b46dc67 admin admin] PATH_INFO: `/v3/users/cfce2b8a66354c9888279fdd8eca11d6` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:32.071434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33bc8e-8dbd-4e44-8823-456c8b46dc67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfce2b8a66354c9888279fdd8eca11d6)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:32.073273 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33bc8e-8dbd-4e44-8823-456c8b46dc67 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:32.076725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 762/1538] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:32 2022] PUT /identity/v3/projects/7ffd679007cd4fc8ab55ddf9cf6ca94f/users/1f2f0682f3d34de2b9ae5f4371550ff4/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:32.087117 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202bf5d-3e2e-43c2-9b70-ffc8cde1ffbe None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:32.087401 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202bf5d-3e2e-43c2-9b70-ffc8cde1ffbe None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:32.087710 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202bf5d-3e2e-43c2-9b70-ffc8cde1ffbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:32.120260 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-c202bf5d-3e2e-43c2-9b70-ffc8cde1ffbe None None] MFA Rules not processed for user `1f2f0682f3d34de2b9ae5f4371550ff4`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:32.122012 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c202bf5d-3e2e-43c2-9b70-ffc8cde1ffbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:32.131983 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ba33bc8e-8dbd-4e44-8823-456c8b46dc67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfce2b8a66354c9888279fdd8eca11d6'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:32.145371 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ba33bc8e-8dbd-4e44-8823-456c8b46dc67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfce2b8a66354c9888279fdd8eca11d6'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:32.154814 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ba33bc8e-8dbd-4e44-8823-456c8b46dc67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfce2b8a66354c9888279fdd8eca11d6'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:32.155495 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ba33bc8e-8dbd-4e44-8823-456c8b46dc67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfce2b8a66354c9888279fdd8eca11d6'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:32.167002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ba33bc8e-8dbd-4e44-8823-456c8b46dc67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfce2b8a66354c9888279fdd8eca11d6'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:32.178737 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 777/1539] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:32 2022] DELETE /identity/v3/users/cfce2b8a66354c9888279fdd8eca11d6 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:32.180427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a17dd-bb06-441b-a832-e03fe15e64d1 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:32.180704 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a17dd-bb06-441b-a832-e03fe15e64d1 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:32.181017 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a17dd-bb06-441b-a832-e03fe15e64d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:32.200508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 763/1540] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:32.205518 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202bf5d-3e2e-43c2-9b70-ffc8cde1ffbe None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:32.207382 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c202bf5d-3e2e-43c2-9b70-ffc8cde1ffbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:32.208398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-759a17dd-bb06-441b-a832-e03fe15e64d1 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:32.209951 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-759a17dd-bb06-441b-a832-e03fe15e64d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:32.279903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 778/1541] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:32.293559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a17dd-bb06-441b-a832-e03fe15e64d1 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:32.294722 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202bf5d-3e2e-43c2-9b70-ffc8cde1ffbe None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:32.296138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-759a17dd-bb06-441b-a832-e03fe15e64d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:32.301730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb273de-fa8a-4c56-b9fd-3adb0296e87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:32.302360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb273de-fa8a-4c56-b9fd-3adb0296e87c admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:32.302642 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb273de-fa8a-4c56-b9fd-3adb0296e87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:32.302914 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb273de-fa8a-4c56-b9fd-3adb0296e87c admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:32.304088 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb273de-fa8a-4c56-b9fd-3adb0296e87c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:32.306938 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb273de-fa8a-4c56-b9fd-3adb0296e87c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:32.332628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:32.346562 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 764/1542] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:32 2022] POST /identity/v3/projects => generated 387 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:32.352052 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb273de-fa8a-4c56-b9fd-3adb0296e87c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:32.383534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a17dd-bb06-441b-a832-e03fe15e64d1 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:32.428776 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:32.432669 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb273de-fa8a-4c56-b9fd-3adb0296e87c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:32.438879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc7345-a0de-44f7-aff2-394a133d89e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:32.439361 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85011a24-d6e4-408e-8b0f-358632ce7cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:32.439500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc7345-a0de-44f7-aff2-394a133d89e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:32.439791 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc7345-a0de-44f7-aff2-394a133d89e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:32.440002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85011a24-d6e4-408e-8b0f-358632ce7cac service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:32.440096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc7345-a0de-44f7-aff2-394a133d89e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:32.440281 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85011a24-d6e4-408e-8b0f-358632ce7cac service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:32.440601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85011a24-d6e4-408e-8b0f-358632ce7cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:32.440707 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc7345-a0de-44f7-aff2-394a133d89e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:32.442596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc7345-a0de-44f7-aff2-394a133d89e4 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:32.449831 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85011a24-d6e4-408e-8b0f-358632ce7cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:32.451270 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85011a24-d6e4-408e-8b0f-358632ce7cac service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:32.461007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 779/1543] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:32.465927 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 765/1544] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:32 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:32.468556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85011a24-d6e4-408e-8b0f-358632ce7cac service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:32.479157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85011a24-d6e4-408e-8b0f-358632ce7cac service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:32.483976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d1b52-94ff-44e1-99c1-c0f1cf45d32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:32.484758 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d1b52-94ff-44e1-99c1-c0f1cf45d32a admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:32.485152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d1b52-94ff-44e1-99c1-c0f1cf45d32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:32.485432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d1b52-94ff-44e1-99c1-c0f1cf45d32a admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:32.486122 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d1b52-94ff-44e1-99c1-c0f1cf45d32a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:32.488390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d1b52-94ff-44e1-99c1-c0f1cf45d32a admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:32.501160 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 780/1545] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:32.502578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc7345-a0de-44f7-aff2-394a133d89e4 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:32.513460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc7345-a0de-44f7-aff2-394a133d89e4 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:32.518153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d9d4e-cac5-4e91-aee7-9e3cfcedee03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:32.518782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d9d4e-cac5-4e91-aee7-9e3cfcedee03 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:32.519067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d9d4e-cac5-4e91-aee7-9e3cfcedee03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:32.519344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d9d4e-cac5-4e91-aee7-9e3cfcedee03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:32.520013 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d9d4e-cac5-4e91-aee7-9e3cfcedee03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:32.521611 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d9d4e-cac5-4e91-aee7-9e3cfcedee03 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:32.532407 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 766/1546] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:32.537717 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d1b52-94ff-44e1-99c1-c0f1cf45d32a admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:32.548984 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d1b52-94ff-44e1-99c1-c0f1cf45d32a admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:32.554772 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57094a3b-45a8-4154-8e77-a9efca3baad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:32.555599 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57094a3b-45a8-4154-8e77-a9efca3baad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:32.555909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57094a3b-45a8-4154-8e77-a9efca3baad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:32.556321 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57094a3b-45a8-4154-8e77-a9efca3baad3 admin admin] PATH_INFO: `/v3/projects/2ec803836f124dc7b18c834dafd23599/users/37ee2943603348c8a65f136e588e2421/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:32.574044 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57094a3b-45a8-4154-8e77-a9efca3baad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ec803836f124dc7b18c834dafd23599, user_id=37ee2943603348c8a65f136e588e2421, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:32.577548 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57094a3b-45a8-4154-8e77-a9efca3baad3 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:32.608941 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 781/1547] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:32 2022] PUT /identity/v3/projects/2ec803836f124dc7b18c834dafd23599/users/37ee2943603348c8a65f136e588e2421/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:32.611775 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7dd7c-ab81-42ee-bfde-86ee55b311a8 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:32.612134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7dd7c-ab81-42ee-bfde-86ee55b311a8 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:32.612833 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7dd7c-ab81-42ee-bfde-86ee55b311a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:32.644272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-3de7dd7c-ab81-42ee-bfde-86ee55b311a8 None None] MFA Rules not processed for user `37ee2943603348c8a65f136e588e2421`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:32.648487 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3de7dd7c-ab81-42ee-bfde-86ee55b311a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:32.748186 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 767/1548] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:32.753105 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57094a3b-45a8-4154-8e77-a9efca3baad3 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:32.808602 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7dd7c-ab81-42ee-bfde-86ee55b311a8 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:32.810094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3de7dd7c-ab81-42ee-bfde-86ee55b311a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:32.842933 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57094a3b-45a8-4154-8e77-a9efca3baad3 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:32.847838 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03285ddf-2bf9-4793-9072-2cd4874527d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:32.848523 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03285ddf-2bf9-4793-9072-2cd4874527d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:32.848801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03285ddf-2bf9-4793-9072-2cd4874527d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:32.849070 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03285ddf-2bf9-4793-9072-2cd4874527d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:32.849711 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03285ddf-2bf9-4793-9072-2cd4874527d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:32.851703 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03285ddf-2bf9-4793-9072-2cd4874527d2 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:32.884558 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 782/1549] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:32 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:32.888882 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03285ddf-2bf9-4793-9072-2cd4874527d2 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:32.932654 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:32.965849 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03285ddf-2bf9-4793-9072-2cd4874527d2 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:32.971503 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc5cf0-ffb1-4df2-ada0-63f85de78681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:32.972755 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc5cf0-ffb1-4df2-ada0-63f85de78681 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:32.973036 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc5cf0-ffb1-4df2-ada0-63f85de78681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:32.973310 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc5cf0-ffb1-4df2-ada0-63f85de78681 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:32.974069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc5cf0-ffb1-4df2-ada0-63f85de78681 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:32.976406 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc5cf0-ffb1-4df2-ada0-63f85de78681 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:33.007324 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 783/1550] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:32 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:21:33.009663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc5cf0-ffb1-4df2-ada0-63f85de78681 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:33.024827 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7dd7c-ab81-42ee-bfde-86ee55b311a8 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:33.030174 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06308b-6638-4c51-ac06-c33284e2274c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:33.031656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06308b-6638-4c51-ac06-c33284e2274c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:33.031656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06308b-6638-4c51-ac06-c33284e2274c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:33.031656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06308b-6638-4c51-ac06-c33284e2274c admin admin] PATH_INFO: `/v3/projects/06c62ce86b224a0c9249b0b7f5477979` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:33.032643 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06308b-6638-4c51-ac06-c33284e2274c admin admin] RBAC: Authorizing `identity:delete_project(project_id=06c62ce86b224a0c9249b0b7f5477979)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:33.035045 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06308b-6638-4c51-ac06-c33284e2274c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:33.038159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc5cf0-ffb1-4df2-ada0-63f85de78681 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:33.043399 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34628ed9-b986-4090-96f8-9884fb3436c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:33.044045 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34628ed9-b986-4090-96f8-9884fb3436c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:33.044326 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34628ed9-b986-4090-96f8-9884fb3436c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:33.044597 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34628ed9-b986-4090-96f8-9884fb3436c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:33.045236 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34628ed9-b986-4090-96f8-9884fb3436c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:33.046849 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34628ed9-b986-4090-96f8-9884fb3436c7 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:33.066442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 784/1551] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:21:33.070612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34628ed9-b986-4090-96f8-9884fb3436c7 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:33.085935 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34628ed9-b986-4090-96f8-9884fb3436c7 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:33.094422 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac7b09-1ad7-4325-83ea-2b5060020673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:33.095946 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac7b09-1ad7-4325-83ea-2b5060020673 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:33.097372 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac7b09-1ad7-4325-83ea-2b5060020673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:33.098966 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac7b09-1ad7-4325-83ea-2b5060020673 admin admin] PATH_INFO: `/v3/projects/a3788324495443d59834c9aa6b484bbc/users/05214c00a99b495c9652073b75447c16/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:33.127901 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-8a06308b-6638-4c51-ac06-c33284e2274c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06c62ce86b224a0c9249b0b7f5477979', 'request_id': 'req-8a06308b-6638-4c51-ac06-c33284e2274c'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:33.139550 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-8a06308b-6638-4c51-ac06-c33284e2274c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06c62ce86b224a0c9249b0b7f5477979', 'request_id': 'req-8a06308b-6638-4c51-ac06-c33284e2274c'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:33.148026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac7b09-1ad7-4325-83ea-2b5060020673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3788324495443d59834c9aa6b484bbc, user_id=05214c00a99b495c9652073b75447c16, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:33.153756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac7b09-1ad7-4325-83ea-2b5060020673 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:33.156211 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-8a06308b-6638-4c51-ac06-c33284e2274c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06c62ce86b224a0c9249b0b7f5477979', 'request_id': 'req-8a06308b-6638-4c51-ac06-c33284e2274c'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:33.169008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:33.204270 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 785/1552] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:33 2022] PUT /identity/v3/projects/a3788324495443d59834c9aa6b484bbc/users/05214c00a99b495c9652073b75447c16/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:33.210914 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac7b09-1ad7-4325-83ea-2b5060020673 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:33.214376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-8a06308b-6638-4c51-ac06-c33284e2274c admin admin] The token cache is being invalidate because project 06c62ce86b224a0c9249b0b7f5477979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:33.215884 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-8a06308b-6638-4c51-ac06-c33284e2274c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:33.230853 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-8a06308b-6638-4c51-ac06-c33284e2274c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:33.235651 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 768/1553] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:32 2022] DELETE /identity/v3/projects/06c62ce86b224a0c9249b0b7f5477979 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:33.237052 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06308b-6638-4c51-ac06-c33284e2274c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:33.245476 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8a06308b-6638-4c51-ac06-c33284e2274c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:33.377805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac7b09-1ad7-4325-83ea-2b5060020673 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:33.393304 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0acbe-847a-47c0-86c8-cd1b80f44260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:33.395854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0acbe-847a-47c0-86c8-cd1b80f44260 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:33.396235 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0acbe-847a-47c0-86c8-cd1b80f44260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:33.398030 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0acbe-847a-47c0-86c8-cd1b80f44260 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:33.398680 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0acbe-847a-47c0-86c8-cd1b80f44260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:33.404031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0acbe-847a-47c0-86c8-cd1b80f44260 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:33.430976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 786/1554] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:33.434762 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0acbe-847a-47c0-86c8-cd1b80f44260 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:33.438533 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-16f0acbe-847a-47c0-86c8-cd1b80f44260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:33.476657 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0acbe-847a-47c0-86c8-cd1b80f44260 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:33.497613 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4b0b2-b159-41d4-be35-1f860a88f987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:33.497852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06308b-6638-4c51-ac06-c33284e2274c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:33.498076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4b0b2-b159-41d4-be35-1f860a88f987 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:33.500769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4b0b2-b159-41d4-be35-1f860a88f987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:33.503022 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4b0b2-b159-41d4-be35-1f860a88f987 admin admin] PATH_INFO: `/v3/projects/a3788324495443d59834c9aa6b484bbc/users/05214c00a99b495c9652073b75447c16/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:33.512131 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4b0b2-b159-41d4-be35-1f860a88f987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3788324495443d59834c9aa6b484bbc, user_id=05214c00a99b495c9652073b75447c16, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:33.512991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88195ae4-6542-4781-a233-30a9ff223481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:33.515514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88195ae4-6542-4781-a233-30a9ff223481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:33.516001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88195ae4-6542-4781-a233-30a9ff223481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:33.516375 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88195ae4-6542-4781-a233-30a9ff223481 admin admin] PATH_INFO: `/v3/users/0d9429fda39545a09923752ba279a02c` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:33.518037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88195ae4-6542-4781-a233-30a9ff223481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d9429fda39545a09923752ba279a02c)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:33.521189 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88195ae4-6542-4781-a233-30a9ff223481 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:33.530712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4b0b2-b159-41d4-be35-1f860a88f987 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:33.576546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 787/1555] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:33 2022] PUT /identity/v3/projects/a3788324495443d59834c9aa6b484bbc/users/05214c00a99b495c9652073b75447c16/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:33.588846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba02ca-cbdd-4868-a8ca-bbafbb95891e None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:33.589181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba02ca-cbdd-4868-a8ca-bbafbb95891e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:33.589666 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba02ca-cbdd-4868-a8ca-bbafbb95891e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:33.627361 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-88195ae4-6542-4781-a233-30a9ff223481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d9429fda39545a09923752ba279a02c'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:33.628951 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-88195ae4-6542-4781-a233-30a9ff223481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d9429fda39545a09923752ba279a02c'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:33.665947 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-88195ae4-6542-4781-a233-30a9ff223481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d9429fda39545a09923752ba279a02c'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:33.672305 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-56ba02ca-cbdd-4868-a8ca-bbafbb95891e None None] MFA Rules not processed for user `05214c00a99b495c9652073b75447c16`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:33.673618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-56ba02ca-cbdd-4868-a8ca-bbafbb95891e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:33.684813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-88195ae4-6542-4781-a233-30a9ff223481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d9429fda39545a09923752ba279a02c'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:33.685548 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-88195ae4-6542-4781-a233-30a9ff223481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d9429fda39545a09923752ba279a02c'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:33.714310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 769/1556] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:33 2022] DELETE /identity/v3/users/0d9429fda39545a09923752ba279a02c => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:33.797672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:33.834365 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 788/1557] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:33 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:33.844215 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88195ae4-6542-4781-a233-30a9ff223481 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:33.846263 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-88195ae4-6542-4781-a233-30a9ff223481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:33.961984 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88195ae4-6542-4781-a233-30a9ff223481 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:33.967471 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adc04a-bb2a-4299-8b08-dfa0fe80a9d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:33.968276 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adc04a-bb2a-4299-8b08-dfa0fe80a9d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:33.968634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adc04a-bb2a-4299-8b08-dfa0fe80a9d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:33.969000 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adc04a-bb2a-4299-8b08-dfa0fe80a9d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:34.028228 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:34.084702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adc04a-bb2a-4299-8b08-dfa0fe80a9d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:34.086888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adc04a-bb2a-4299-8b08-dfa0fe80a9d4 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:34.108021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 770/1558] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:33 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:34.895603 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba02ca-cbdd-4868-a8ca-bbafbb95891e None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:34.990620 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adc04a-bb2a-4299-8b08-dfa0fe80a9d4 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:34.994454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-91adc04a-bb2a-4299-8b08-dfa0fe80a9d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:35.054990 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba02ca-cbdd-4868-a8ca-bbafbb95891e None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:35.058639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c965fc-e30f-44fb-b90c-fd946df6c23b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:35.059258 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c965fc-e30f-44fb-b90c-fd946df6c23b service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:35.059895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c965fc-e30f-44fb-b90c-fd946df6c23b service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:35.059895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c965fc-e30f-44fb-b90c-fd946df6c23b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:35.061631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-92c965fc-e30f-44fb-b90c-fd946df6c23b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:35.093024 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adc04a-bb2a-4299-8b08-dfa0fe80a9d4 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:35.099095 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df4b77-4712-41c2-86e4-6652b53f708e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:35.100208 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df4b77-4712-41c2-86e4-6652b53f708e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:35.100208 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df4b77-4712-41c2-86e4-6652b53f708e admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:35.100660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df4b77-4712-41c2-86e4-6652b53f708e admin admin] PATH_INFO: `/v3/users/78e948ff9b12494dbbc752263b6c5405` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:35.101993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df4b77-4712-41c2-86e4-6652b53f708e admin admin] RBAC: Authorizing `identity:delete_user(user_id=78e948ff9b12494dbbc752263b6c5405)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:35.104483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df4b77-4712-41c2-86e4-6652b53f708e admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:35.204289 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c965fc-e30f-44fb-b90c-fd946df6c23b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:35.204962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-92df4b77-4712-41c2-86e4-6652b53f708e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78e948ff9b12494dbbc752263b6c5405'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:35.207475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c965fc-e30f-44fb-b90c-fd946df6c23b service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:35.207942 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-92df4b77-4712-41c2-86e4-6652b53f708e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78e948ff9b12494dbbc752263b6c5405'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:35.241482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-92df4b77-4712-41c2-86e4-6652b53f708e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78e948ff9b12494dbbc752263b6c5405'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:35.251802 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 789/1559] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:35.254304 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c965fc-e30f-44fb-b90c-fd946df6c23b service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:35.255958 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-92df4b77-4712-41c2-86e4-6652b53f708e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78e948ff9b12494dbbc752263b6c5405'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:35.257009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-92df4b77-4712-41c2-86e4-6652b53f708e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78e948ff9b12494dbbc752263b6c5405'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:35.259282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-92c965fc-e30f-44fb-b90c-fd946df6c23b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:35.286972 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 771/1560] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:34 2022] DELETE /identity/v3/users/78e948ff9b12494dbbc752263b6c5405 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:35.291450 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8467f4-2642-4768-b76c-cffb3b81cdb9 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:35.292520 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8467f4-2642-4768-b76c-cffb3b81cdb9 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:35.293331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8467f4-2642-4768-b76c-cffb3b81cdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:35.295436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c965fc-e30f-44fb-b90c-fd946df6c23b service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:35.337099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-ee8467f4-2642-4768-b76c-cffb3b81cdb9 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:35.338373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ee8467f4-2642-4768-b76c-cffb3b81cdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:35.436117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1347c6-8e22-4ff0-a643-d6e6319522a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:35.439753 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1347c6-8e22-4ff0-a643-d6e6319522a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:35.440057 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1347c6-8e22-4ff0-a643-d6e6319522a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:35.440332 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1347c6-8e22-4ff0-a643-d6e6319522a9 admin admin] PATH_INFO: `/v3/projects/4481acf2875f49e7a508d223fa6714a3` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:35.443207 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1347c6-8e22-4ff0-a643-d6e6319522a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4481acf2875f49e7a508d223fa6714a3)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:35.447370 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1347c6-8e22-4ff0-a643-d6e6319522a9 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:35.447492 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:35.492526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 772/1561] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:35.508118 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8467f4-2642-4768-b76c-cffb3b81cdb9 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:35.510354 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ee8467f4-2642-4768-b76c-cffb3b81cdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:35.588103 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-cd1347c6-8e22-4ff0-a643-d6e6319522a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4481acf2875f49e7a508d223fa6714a3', 'request_id': 'req-cd1347c6-8e22-4ff0-a643-d6e6319522a9'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:35.589557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:35.595174 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-cd1347c6-8e22-4ff0-a643-d6e6319522a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4481acf2875f49e7a508d223fa6714a3', 'request_id': 'req-cd1347c6-8e22-4ff0-a643-d6e6319522a9'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:35.606458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-cd1347c6-8e22-4ff0-a643-d6e6319522a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4481acf2875f49e7a508d223fa6714a3', 'request_id': 'req-cd1347c6-8e22-4ff0-a643-d6e6319522a9'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:35.620300 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8467f4-2642-4768-b76c-cffb3b81cdb9 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:35.623795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-cd1347c6-8e22-4ff0-a643-d6e6319522a9 admin admin] The token cache is being invalidate because project 4481acf2875f49e7a508d223fa6714a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:35.624093 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-cd1347c6-8e22-4ff0-a643-d6e6319522a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:35.624756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-cd1347c6-8e22-4ff0-a643-d6e6319522a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:35.626607 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 790/1562] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:35 2022] DELETE /identity/v3/projects/4481acf2875f49e7a508d223fa6714a3 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:35.631587 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc6a2f-2b27-434b-b01f-56eca4d48555 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:35.631778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc6a2f-2b27-434b-b01f-56eca4d48555 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:35.632793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc6a2f-2b27-434b-b01f-56eca4d48555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:35.700940 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-9cfc6a2f-2b27-434b-b01f-56eca4d48555 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:35.702280 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9cfc6a2f-2b27-434b-b01f-56eca4d48555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:35.734392 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6137f-4e70-4e00-b206-74aa9905191b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:35.751862 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6137f-4e70-4e00-b206-74aa9905191b service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:35.752221 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6137f-4e70-4e00-b206-74aa9905191b service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:35.752501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6137f-4e70-4e00-b206-74aa9905191b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:35.763837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-81d6137f-4e70-4e00-b206-74aa9905191b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:35.811351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 791/1563] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:35.820622 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07e608-397f-44ed-9bbd-d22888c330f4 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:35.821184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07e608-397f-44ed-9bbd-d22888c330f4 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:35.821885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07e608-397f-44ed-9bbd-d22888c330f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:35.890757 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-8f07e608-397f-44ed-9bbd-d22888c330f4 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:35.895485 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8f07e608-397f-44ed-9bbd-d22888c330f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:35.927854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 792/1564] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:35.933284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07e608-397f-44ed-9bbd-d22888c330f4 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:35.953847 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07e608-397f-44ed-9bbd-d22888c330f4 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:35.958094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6137f-4e70-4e00-b206-74aa9905191b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:35.959745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6137f-4e70-4e00-b206-74aa9905191b service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:35.967415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62711cd2-341c-490f-bd80-be99f53a9558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:35.968330 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62711cd2-341c-490f-bd80-be99f53a9558 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:35.968803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62711cd2-341c-490f-bd80-be99f53a9558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:35.969288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62711cd2-341c-490f-bd80-be99f53a9558 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:35.970127 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62711cd2-341c-490f-bd80-be99f53a9558 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:21:35.970605 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62711cd2-341c-490f-bd80-be99f53a9558 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:35.973354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62711cd2-341c-490f-bd80-be99f53a9558 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:35.973789 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 773/1565] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:35.996885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 793/1566] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:21:36.000936 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6137f-4e70-4e00-b206-74aa9905191b service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:36.015409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6137f-4e70-4e00-b206-74aa9905191b service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:36.039880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace00069-675f-45cb-be2e-32d4001142cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:36.040763 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace00069-675f-45cb-be2e-32d4001142cb admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:36.041147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace00069-675f-45cb-be2e-32d4001142cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:36.041432 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace00069-675f-45cb-be2e-32d4001142cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:36.042143 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace00069-675f-45cb-be2e-32d4001142cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:36.044957 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace00069-675f-45cb-be2e-32d4001142cb admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:36.093890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 774/1567] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:35 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:36.098562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62711cd2-341c-490f-bd80-be99f53a9558 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:36.119166 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62711cd2-341c-490f-bd80-be99f53a9558 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:36.209696 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1912e9-40c3-4543-be68-6baaedd8edfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:36.211283 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1912e9-40c3-4543-be68-6baaedd8edfa admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:36.211283 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1912e9-40c3-4543-be68-6baaedd8edfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:36.211283 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1912e9-40c3-4543-be68-6baaedd8edfa admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:36.211543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1912e9-40c3-4543-be68-6baaedd8edfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:36.213880 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1912e9-40c3-4543-be68-6baaedd8edfa admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:36.244117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 794/1568] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:36 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:36.248121 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace00069-675f-45cb-be2e-32d4001142cb admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:36.263049 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace00069-675f-45cb-be2e-32d4001142cb admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:36.269024 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f15fa-6cd4-48d6-a37c-861fdc928693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:36.270071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f15fa-6cd4-48d6-a37c-861fdc928693 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:36.270556 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f15fa-6cd4-48d6-a37c-861fdc928693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:36.271612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f15fa-6cd4-48d6-a37c-861fdc928693 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:36.272729 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f15fa-6cd4-48d6-a37c-861fdc928693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:36.274647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f15fa-6cd4-48d6-a37c-861fdc928693 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:36.290130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 775/1569] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:36.297237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1912e9-40c3-4543-be68-6baaedd8edfa admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:36.309307 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1912e9-40c3-4543-be68-6baaedd8edfa admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:36.314973 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06b8ba-7fe8-4f11-b082-019dd66a58c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:36.316561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06b8ba-7fe8-4f11-b082-019dd66a58c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:36.317192 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06b8ba-7fe8-4f11-b082-019dd66a58c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:36.317642 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06b8ba-7fe8-4f11-b082-019dd66a58c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:36.318456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06b8ba-7fe8-4f11-b082-019dd66a58c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:36.320780 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06b8ba-7fe8-4f11-b082-019dd66a58c6 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:36.335627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 795/1570] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:36.339210 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f15fa-6cd4-48d6-a37c-861fdc928693 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:36.351652 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f15fa-6cd4-48d6-a37c-861fdc928693 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:36.358071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0226a4-762a-44ae-be05-198dda4c0bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:36.358942 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0226a4-762a-44ae-be05-198dda4c0bd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:36.359426 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0226a4-762a-44ae-be05-198dda4c0bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:36.359933 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0226a4-762a-44ae-be05-198dda4c0bd6 admin admin] PATH_INFO: `/v3/projects/97aa42626a4b42c58f69fc92f6b63a49/users/c071160af585481dadb46225509f1520/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:36.398814 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0226a4-762a-44ae-be05-198dda4c0bd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97aa42626a4b42c58f69fc92f6b63a49, user_id=c071160af585481dadb46225509f1520, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:36.407366 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0226a4-762a-44ae-be05-198dda4c0bd6 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:36.415082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:36.442121 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 776/1571] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:36 2022] PUT /identity/v3/projects/97aa42626a4b42c58f69fc92f6b63a49/users/c071160af585481dadb46225509f1520/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:36.453165 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210522f-0871-4b3e-8696-477d72df42ec None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:36.453601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210522f-0871-4b3e-8696-477d72df42ec None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:36.453770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210522f-0871-4b3e-8696-477d72df42ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:36.488206 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-c210522f-0871-4b3e-8696-477d72df42ec None None] MFA Rules not processed for user `c071160af585481dadb46225509f1520`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:36.490279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-c210522f-0871-4b3e-8696-477d72df42ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:36.540657 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:36.597088 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 796/1572] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:36.608571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0226a4-762a-44ae-be05-198dda4c0bd6 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:36.610491 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2a0226a4-762a-44ae-be05-198dda4c0bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:36.682048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0226a4-762a-44ae-be05-198dda4c0bd6 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:36.687887 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e53e90-1779-47a7-b0c3-58c3ec886df4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:36.689119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e53e90-1779-47a7-b0c3-58c3ec886df4 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:36.689481 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e53e90-1779-47a7-b0c3-58c3ec886df4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:36.690250 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e53e90-1779-47a7-b0c3-58c3ec886df4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:36.705638 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e53e90-1779-47a7-b0c3-58c3ec886df4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:36.707878 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e53e90-1779-47a7-b0c3-58c3ec886df4 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:36.732773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 777/1573] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:36.922184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210522f-0871-4b3e-8696-477d72df42ec None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:36.924413 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-c210522f-0871-4b3e-8696-477d72df42ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:37.090684 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210522f-0871-4b3e-8696-477d72df42ec None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:37.102463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc615580-8c45-4e33-bbf2-631fe56ffcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:37.103394 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc615580-8c45-4e33-bbf2-631fe56ffcb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:37.105341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc615580-8c45-4e33-bbf2-631fe56ffcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:37.105884 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc615580-8c45-4e33-bbf2-631fe56ffcb9 admin admin] PATH_INFO: `/v3/projects/4ecf4e8fc064433c95078468a33b1227` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:37.108494 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc615580-8c45-4e33-bbf2-631fe56ffcb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ecf4e8fc064433c95078468a33b1227)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:37.113764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc615580-8c45-4e33-bbf2-631fe56ffcb9 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:37.269030 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-bc615580-8c45-4e33-bbf2-631fe56ffcb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ecf4e8fc064433c95078468a33b1227', 'request_id': 'req-bc615580-8c45-4e33-bbf2-631fe56ffcb9'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:37.291244 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-bc615580-8c45-4e33-bbf2-631fe56ffcb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ecf4e8fc064433c95078468a33b1227', 'request_id': 'req-bc615580-8c45-4e33-bbf2-631fe56ffcb9'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:37.326514 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-bc615580-8c45-4e33-bbf2-631fe56ffcb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ecf4e8fc064433c95078468a33b1227', 'request_id': 'req-bc615580-8c45-4e33-bbf2-631fe56ffcb9'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:37.350228 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-bc615580-8c45-4e33-bbf2-631fe56ffcb9 admin admin] The token cache is being invalidate because project 4ecf4e8fc064433c95078468a33b1227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:37.350647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-bc615580-8c45-4e33-bbf2-631fe56ffcb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:37.352183 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-bc615580-8c45-4e33-bbf2-631fe56ffcb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:37.354830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 797/1574] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:36 2022] DELETE /identity/v3/projects/4ecf4e8fc064433c95078468a33b1227 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:37.358958 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e53e90-1779-47a7-b0c3-58c3ec886df4 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:37.389389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e53e90-1779-47a7-b0c3-58c3ec886df4 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:37.562316 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502d597-41fe-4bcf-968e-f838296753ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:37.563055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502d597-41fe-4bcf-968e-f838296753ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:37.563364 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502d597-41fe-4bcf-968e-f838296753ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:37.563672 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502d597-41fe-4bcf-968e-f838296753ea admin admin] PATH_INFO: `/v3/users/58ea4271935c4c468543ae55d0e53426` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:37.564859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502d597-41fe-4bcf-968e-f838296753ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=58ea4271935c4c468543ae55d0e53426)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:37.567294 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502d597-41fe-4bcf-968e-f838296753ea admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:37.704675 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e502d597-41fe-4bcf-968e-f838296753ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58ea4271935c4c468543ae55d0e53426'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:37.705453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e502d597-41fe-4bcf-968e-f838296753ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58ea4271935c4c468543ae55d0e53426'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:37.763934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e502d597-41fe-4bcf-968e-f838296753ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58ea4271935c4c468543ae55d0e53426'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:37.772836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e502d597-41fe-4bcf-968e-f838296753ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58ea4271935c4c468543ae55d0e53426'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:37.774423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e502d597-41fe-4bcf-968e-f838296753ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58ea4271935c4c468543ae55d0e53426'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:37.808079 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 778/1575] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:37 2022] DELETE /identity/v3/users/58ea4271935c4c468543ae55d0e53426 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:39.109910 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc615580-8c45-4e33-bbf2-631fe56ffcb9 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:39.111788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-bc615580-8c45-4e33-bbf2-631fe56ffcb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:39.125120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc615580-8c45-4e33-bbf2-631fe56ffcb9 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:39.194816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:39.201892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3a1c4-2e12-4d1f-89e8-d29339ce2045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:39.202911 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3a1c4-2e12-4d1f-89e8-d29339ce2045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:39.203400 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3a1c4-2e12-4d1f-89e8-d29339ce2045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:39.204078 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3a1c4-2e12-4d1f-89e8-d29339ce2045 admin admin] PATH_INFO: `/v3/projects/3c27c5ae7e3d477996c064ad6b208329` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:39.205813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3a1c4-2e12-4d1f-89e8-d29339ce2045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c27c5ae7e3d477996c064ad6b208329)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:39.208318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3a1c4-2e12-4d1f-89e8-d29339ce2045 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:39.261478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d5d3a1c4-2e12-4d1f-89e8-d29339ce2045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c27c5ae7e3d477996c064ad6b208329', 'request_id': 'req-d5d3a1c4-2e12-4d1f-89e8-d29339ce2045'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:39.267958 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d5d3a1c4-2e12-4d1f-89e8-d29339ce2045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c27c5ae7e3d477996c064ad6b208329', 'request_id': 'req-d5d3a1c4-2e12-4d1f-89e8-d29339ce2045'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:39.286475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d5d3a1c4-2e12-4d1f-89e8-d29339ce2045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c27c5ae7e3d477996c064ad6b208329', 'request_id': 'req-d5d3a1c4-2e12-4d1f-89e8-d29339ce2045'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:39.307068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d5d3a1c4-2e12-4d1f-89e8-d29339ce2045 admin admin] The token cache is being invalidate because project 3c27c5ae7e3d477996c064ad6b208329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:39.307607 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d5d3a1c4-2e12-4d1f-89e8-d29339ce2045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:39.308584 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d5d3a1c4-2e12-4d1f-89e8-d29339ce2045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:39.310583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 798/1576] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:39 2022] DELETE /identity/v3/projects/3c27c5ae7e3d477996c064ad6b208329 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:39.316138 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b220ce-ff1b-4792-b433-3346fbef761b None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:39.316648 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b220ce-ff1b-4792-b433-3346fbef761b None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:39.316987 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b220ce-ff1b-4792-b433-3346fbef761b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:39.338409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-d0b220ce-ff1b-4792-b433-3346fbef761b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:39.339678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d0b220ce-ff1b-4792-b433-3346fbef761b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:39.367243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:39.475140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 779/1577] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:39.483481 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c074aef-af94-4385-917a-8efce98063ec None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:39.483944 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c074aef-af94-4385-917a-8efce98063ec None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:39.484279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c074aef-af94-4385-917a-8efce98063ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:39.517542 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-4c074aef-af94-4385-917a-8efce98063ec None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:39.519384 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-4c074aef-af94-4385-917a-8efce98063ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:39.528818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 799/1578] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:39.533797 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b220ce-ff1b-4792-b433-3346fbef761b None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:39.546342 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b220ce-ff1b-4792-b433-3346fbef761b None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:39.552987 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3796a44-ccff-4130-9f39-67aca3c7c467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:39.553628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3796a44-ccff-4130-9f39-67aca3c7c467 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:39.553981 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3796a44-ccff-4130-9f39-67aca3c7c467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:39.554561 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3796a44-ccff-4130-9f39-67aca3c7c467 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:39.555282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3796a44-ccff-4130-9f39-67aca3c7c467 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:21:39.555676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3796a44-ccff-4130-9f39-67aca3c7c467 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:39.558117 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3796a44-ccff-4130-9f39-67aca3c7c467 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:39.571609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 780/1579] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:21:39.575735 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c074aef-af94-4385-917a-8efce98063ec None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:39.586707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c074aef-af94-4385-917a-8efce98063ec None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:39.591673 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff14d46-27e9-4dc8-81fd-25c619a61947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:39.592273 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff14d46-27e9-4dc8-81fd-25c619a61947 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:39.592646 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff14d46-27e9-4dc8-81fd-25c619a61947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:39.592955 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff14d46-27e9-4dc8-81fd-25c619a61947 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:39.593703 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff14d46-27e9-4dc8-81fd-25c619a61947 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:39.595808 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff14d46-27e9-4dc8-81fd-25c619a61947 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:39.626359 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 800/1580] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:39 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:39.630711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3796a44-ccff-4130-9f39-67aca3c7c467 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:39.640733 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3796a44-ccff-4130-9f39-67aca3c7c467 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:39.700256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846e11c-907b-499b-b9ea-a29df9fa9a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:39.700868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846e11c-907b-499b-b9ea-a29df9fa9a78 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:39.701147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846e11c-907b-499b-b9ea-a29df9fa9a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:39.701426 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846e11c-907b-499b-b9ea-a29df9fa9a78 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:39.702099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d846e11c-907b-499b-b9ea-a29df9fa9a78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:39.704068 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d846e11c-907b-499b-b9ea-a29df9fa9a78 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:39.726885 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 781/1581] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:39 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:39.731184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff14d46-27e9-4dc8-81fd-25c619a61947 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:39.741563 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff14d46-27e9-4dc8-81fd-25c619a61947 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:39.746160 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e2089-750c-4b56-b72d-459bd2b5cec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:39.746761 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e2089-750c-4b56-b72d-459bd2b5cec2 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:39.747041 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e2089-750c-4b56-b72d-459bd2b5cec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:39.747312 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e2089-750c-4b56-b72d-459bd2b5cec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:39.748011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e2089-750c-4b56-b72d-459bd2b5cec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:39.749636 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e2089-750c-4b56-b72d-459bd2b5cec2 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:39.759968 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 801/1582] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:39.764210 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846e11c-907b-499b-b9ea-a29df9fa9a78 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:39.774784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846e11c-907b-499b-b9ea-a29df9fa9a78 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:39.779359 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898a5f5-7bc8-4b62-9304-de4821c2080d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:39.779989 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898a5f5-7bc8-4b62-9304-de4821c2080d admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:39.780269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898a5f5-7bc8-4b62-9304-de4821c2080d admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:39.780639 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898a5f5-7bc8-4b62-9304-de4821c2080d admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:39.781165 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898a5f5-7bc8-4b62-9304-de4821c2080d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:39.782726 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898a5f5-7bc8-4b62-9304-de4821c2080d admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:39.794346 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 782/1583] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:39.798603 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e2089-750c-4b56-b72d-459bd2b5cec2 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:39.808320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e2089-750c-4b56-b72d-459bd2b5cec2 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:39.813120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a062b70-a45b-45f2-8c8a-653a6d9de46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:39.813768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a062b70-a45b-45f2-8c8a-653a6d9de46e admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:39.814048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a062b70-a45b-45f2-8c8a-653a6d9de46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:39.814318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a062b70-a45b-45f2-8c8a-653a6d9de46e admin admin] PATH_INFO: `/v3/projects/882c8513a8124f01bc3ce7c42f4441a9/users/b09e458a860940afb60a00483e88b2aa/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:39.829612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a062b70-a45b-45f2-8c8a-653a6d9de46e admin admin] RBAC: Authorizing `identity:create_grant(project_id=882c8513a8124f01bc3ce7c42f4441a9, user_id=b09e458a860940afb60a00483e88b2aa, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:39.832450 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a062b70-a45b-45f2-8c8a-653a6d9de46e admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:39.845369 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 802/1584] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:39 2022] PUT /identity/v3/projects/882c8513a8124f01bc3ce7c42f4441a9/users/b09e458a860940afb60a00483e88b2aa/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:39.856351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e8324-5cb2-4c37-a59a-eb6324d83407 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:39.856658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e8324-5cb2-4c37-a59a-eb6324d83407 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:39.856962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e8324-5cb2-4c37-a59a-eb6324d83407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:39.913264 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-8e0e8324-5cb2-4c37-a59a-eb6324d83407 None None] MFA Rules not processed for user `b09e458a860940afb60a00483e88b2aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:39.914743 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8e0e8324-5cb2-4c37-a59a-eb6324d83407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:40.007609 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a062b70-a45b-45f2-8c8a-653a6d9de46e admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:40.009654 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-1a062b70-a45b-45f2-8c8a-653a6d9de46e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:40.048124 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 783/1585] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:40.052796 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e8324-5cb2-4c37-a59a-eb6324d83407 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:40.068692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:40.079346 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a062b70-a45b-45f2-8c8a-653a6d9de46e admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:40.082817 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58a50d-53e1-46e0-97c3-7a95a31b7218 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:40.083401 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58a50d-53e1-46e0-97c3-7a95a31b7218 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:40.083707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58a50d-53e1-46e0-97c3-7a95a31b7218 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:40.083970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58a50d-53e1-46e0-97c3-7a95a31b7218 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:40.086345 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2f58a50d-53e1-46e0-97c3-7a95a31b7218 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:40.128281 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e8324-5cb2-4c37-a59a-eb6324d83407 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:40.130541 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:40.138208 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90764305-b70a-42f7-9a43-c74b8ba7898f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:40.139143 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90764305-b70a-42f7-9a43-c74b8ba7898f admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:40.139662 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90764305-b70a-42f7-9a43-c74b8ba7898f admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:40.140198 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90764305-b70a-42f7-9a43-c74b8ba7898f admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:40.140977 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90764305-b70a-42f7-9a43-c74b8ba7898f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:40.144109 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90764305-b70a-42f7-9a43-c74b8ba7898f admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:40.156183 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58a50d-53e1-46e0-97c3-7a95a31b7218 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:40.158031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58a50d-53e1-46e0-97c3-7a95a31b7218 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:40.168653 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 803/1586] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:21:40.193186 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 784/1587] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:40 2022] POST /identity/v3/projects => generated 379 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:21:40.197646 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58a50d-53e1-46e0-97c3-7a95a31b7218 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:40.209290 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58a50d-53e1-46e0-97c3-7a95a31b7218 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:40.275040 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75765190-06eb-479b-af12-bbccb69a5043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:40.275684 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75765190-06eb-479b-af12-bbccb69a5043 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:40.275964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75765190-06eb-479b-af12-bbccb69a5043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:40.276239 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75765190-06eb-479b-af12-bbccb69a5043 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:40.276866 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75765190-06eb-479b-af12-bbccb69a5043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:40.278821 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75765190-06eb-479b-af12-bbccb69a5043 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:40.304402 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 804/1588] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:40 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:21:40.308543 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90764305-b70a-42f7-9a43-c74b8ba7898f admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:40.319929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90764305-b70a-42f7-9a43-c74b8ba7898f admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:40.326029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d402e0-1626-4e30-bcd3-d9d8b25d96f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:40.327006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d402e0-1626-4e30-bcd3-d9d8b25d96f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:40.327006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d402e0-1626-4e30-bcd3-d9d8b25d96f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:40.327430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d402e0-1626-4e30-bcd3-d9d8b25d96f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:40.328611 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d402e0-1626-4e30-bcd3-d9d8b25d96f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:40.330435 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d402e0-1626-4e30-bcd3-d9d8b25d96f0 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:40.346276 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 785/1589] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:40.347722 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75765190-06eb-479b-af12-bbccb69a5043 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:40.359833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75765190-06eb-479b-af12-bbccb69a5043 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:40.366379 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70920c-61b2-4810-984b-8339e9b71139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:40.367048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70920c-61b2-4810-984b-8339e9b71139 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:40.367333 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70920c-61b2-4810-984b-8339e9b71139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:40.368240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70920c-61b2-4810-984b-8339e9b71139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:40.368893 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70920c-61b2-4810-984b-8339e9b71139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:40.370501 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70920c-61b2-4810-984b-8339e9b71139 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:40.383522 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 805/1590] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:21:40.387826 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d402e0-1626-4e30-bcd3-d9d8b25d96f0 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:40.399621 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d402e0-1626-4e30-bcd3-d9d8b25d96f0 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:40.405419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa436a-4796-43cf-85de-e4829b9aa682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:40.406330 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa436a-4796-43cf-85de-e4829b9aa682 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:40.406948 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa436a-4796-43cf-85de-e4829b9aa682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:40.407753 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa436a-4796-43cf-85de-e4829b9aa682 admin admin] PATH_INFO: `/v3/projects/ffb97b7c0032411aa3038d53ad5a0b5c/users/6f12af30bf3c45d2b467627ab0b5ad39/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:40.423673 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa436a-4796-43cf-85de-e4829b9aa682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb97b7c0032411aa3038d53ad5a0b5c, user_id=6f12af30bf3c45d2b467627ab0b5ad39, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:40.428559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa436a-4796-43cf-85de-e4829b9aa682 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:40.445139 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 786/1591] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:40 2022] PUT /identity/v3/projects/ffb97b7c0032411aa3038d53ad5a0b5c/users/6f12af30bf3c45d2b467627ab0b5ad39/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:40.458019 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c83499-6ba8-4c6f-b587-53feafcc7a34 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:40.458305 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c83499-6ba8-4c6f-b587-53feafcc7a34 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:40.458580 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c83499-6ba8-4c6f-b587-53feafcc7a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:40.486986 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-54c83499-6ba8-4c6f-b587-53feafcc7a34 None None] MFA Rules not processed for user `6f12af30bf3c45d2b467627ab0b5ad39`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:40.488466 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-54c83499-6ba8-4c6f-b587-53feafcc7a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:40.569576 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 806/1592] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:40.575361 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa436a-4796-43cf-85de-e4829b9aa682 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:40.639286 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa436a-4796-43cf-85de-e4829b9aa682 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:40.644477 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb36f0-fd3d-4169-aa58-9b508ce54d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:40.645198 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb36f0-fd3d-4169-aa58-9b508ce54d8c admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:40.645477 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb36f0-fd3d-4169-aa58-9b508ce54d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:40.645745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb36f0-fd3d-4169-aa58-9b508ce54d8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:40.646377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cb36f0-fd3d-4169-aa58-9b508ce54d8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:40.648521 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cb36f0-fd3d-4169-aa58-9b508ce54d8c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:40.675603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 787/1593] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:40 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:40.678887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c83499-6ba8-4c6f-b587-53feafcc7a34 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:40.689386 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c83499-6ba8-4c6f-b587-53feafcc7a34 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:40.735168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:40.750660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9bb50-ec6f-475c-af1e-2c595422e55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:40.751278 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9bb50-ec6f-475c-af1e-2c595422e55c admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:40.751556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9bb50-ec6f-475c-af1e-2c595422e55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:40.751902 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9bb50-ec6f-475c-af1e-2c595422e55c admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:40.752556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9bb50-ec6f-475c-af1e-2c595422e55c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:40.754995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9bb50-ec6f-475c-af1e-2c595422e55c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:40.787199 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 807/1594] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:40 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:40.790933 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb36f0-fd3d-4169-aa58-9b508ce54d8c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:40.801909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb36f0-fd3d-4169-aa58-9b508ce54d8c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:40.806461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb8a6d-0bec-4a10-ac35-7ddebc53356f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:40.807062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb8a6d-0bec-4a10-ac35-7ddebc53356f admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:40.807340 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb8a6d-0bec-4a10-ac35-7ddebc53356f admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:40.807639 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb8a6d-0bec-4a10-ac35-7ddebc53356f admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:40.808276 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb8a6d-0bec-4a10-ac35-7ddebc53356f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:40.809859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb8a6d-0bec-4a10-ac35-7ddebc53356f admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:40.820529 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 788/1595] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:40.824630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9bb50-ec6f-475c-af1e-2c595422e55c admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:40.835296 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9bb50-ec6f-475c-af1e-2c595422e55c admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:40.839833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59311697-c791-4174-938d-3ff95665fafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:40.840481 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59311697-c791-4174-938d-3ff95665fafc admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:40.840756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59311697-c791-4174-938d-3ff95665fafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:40.841153 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59311697-c791-4174-938d-3ff95665fafc admin admin] PATH_INFO: `/v3/projects/49bae1df6cf54e24bb4615b115ddb23a/users/a11cb163eb8d495080a918bfba220a8f/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:40.856802 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59311697-c791-4174-938d-3ff95665fafc admin admin] RBAC: Authorizing `identity:create_grant(project_id=49bae1df6cf54e24bb4615b115ddb23a, user_id=a11cb163eb8d495080a918bfba220a8f, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:40.859470 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59311697-c791-4174-938d-3ff95665fafc admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:40.872511 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 808/1596] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:40 2022] PUT /identity/v3/projects/49bae1df6cf54e24bb4615b115ddb23a/users/a11cb163eb8d495080a918bfba220a8f/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:40.876477 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb8a6d-0bec-4a10-ac35-7ddebc53356f admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:40.887094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb8a6d-0bec-4a10-ac35-7ddebc53356f admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:40.928039 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:40.956440 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e8d6f-42bc-401b-8e7c-53a35c7c16a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:40.957054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e8d6f-42bc-401b-8e7c-53a35c7c16a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:40.957336 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e8d6f-42bc-401b-8e7c-53a35c7c16a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:40.957606 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e8d6f-42bc-401b-8e7c-53a35c7c16a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:40.958241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e8d6f-42bc-401b-8e7c-53a35c7c16a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:40.959864 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e8d6f-42bc-401b-8e7c-53a35c7c16a0 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:40.969704 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 789/1597] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:40.973798 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59311697-c791-4174-938d-3ff95665fafc admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:40.983788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59311697-c791-4174-938d-3ff95665fafc admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:40.988825 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb67745-f163-4370-b966-b571723e521a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:40.989480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb67745-f163-4370-b966-b571723e521a admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:40.989757 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb67745-f163-4370-b966-b571723e521a admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:40.990021 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb67745-f163-4370-b966-b571723e521a admin admin] PATH_INFO: `/v3/projects/49bae1df6cf54e24bb4615b115ddb23a/users/a11cb163eb8d495080a918bfba220a8f/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:40.992200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb67745-f163-4370-b966-b571723e521a admin admin] RBAC: Authorizing `identity:create_grant(project_id=49bae1df6cf54e24bb4615b115ddb23a, user_id=a11cb163eb8d495080a918bfba220a8f, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:40.994835 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb67745-f163-4370-b966-b571723e521a admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:41.008665 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 809/1598] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:40 2022] PUT /identity/v3/projects/49bae1df6cf54e24bb4615b115ddb23a/users/a11cb163eb8d495080a918bfba220a8f/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:41.019289 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e2984-4161-45e8-9b8a-511a85a6339b None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:41.019779 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e2984-4161-45e8-9b8a-511a85a6339b None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:41.020128 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e2984-4161-45e8-9b8a-511a85a6339b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:41.049042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-697e2984-4161-45e8-9b8a-511a85a6339b None None] MFA Rules not processed for user `a11cb163eb8d495080a918bfba220a8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:41.050344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-697e2984-4161-45e8-9b8a-511a85a6339b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:41.119042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 790/1599] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:41.130279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb67745-f163-4370-b966-b571723e521a admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:41.131995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-3bb67745-f163-4370-b966-b571723e521a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:41.188396 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb67745-f163-4370-b966-b571723e521a admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:41.191943 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498778c-7493-43aa-afa0-cdaf66632a08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:41.192534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498778c-7493-43aa-afa0-cdaf66632a08 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:41.192808 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498778c-7493-43aa-afa0-cdaf66632a08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:41.193072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498778c-7493-43aa-afa0-cdaf66632a08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:41.201646 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498778c-7493-43aa-afa0-cdaf66632a08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:41.203053 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498778c-7493-43aa-afa0-cdaf66632a08 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:41.212664 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 810/1600] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:41.252620 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e2984-4161-45e8-9b8a-511a85a6339b None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:41.262380 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e2984-4161-45e8-9b8a-511a85a6339b None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:41.265931 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e1727-426b-4c04-8588-609004b2677b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:41.266523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e1727-426b-4c04-8588-609004b2677b service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:41.266798 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e1727-426b-4c04-8588-609004b2677b service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:41.267067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e1727-426b-4c04-8588-609004b2677b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:41.322008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e1727-426b-4c04-8588-609004b2677b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:41.323357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e1727-426b-4c04-8588-609004b2677b service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:41.332187 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 791/1601] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:45.141364 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498778c-7493-43aa-afa0-cdaf66632a08 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:45.142752 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-a498778c-7493-43aa-afa0-cdaf66632a08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:45.219090 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498778c-7493-43aa-afa0-cdaf66632a08 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:45.224150 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c672d3-7bc8-49fe-8393-a141c583e244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:45.224795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c672d3-7bc8-49fe-8393-a141c583e244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:45.225139 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c672d3-7bc8-49fe-8393-a141c583e244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:45.225368 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c672d3-7bc8-49fe-8393-a141c583e244 admin admin] PATH_INFO: `/v3/users/1f2f0682f3d34de2b9ae5f4371550ff4` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:45.226660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c672d3-7bc8-49fe-8393-a141c583e244 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f2f0682f3d34de2b9ae5f4371550ff4)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:45.228384 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c672d3-7bc8-49fe-8393-a141c583e244 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:45.276165 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:45.279270 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-32c672d3-7bc8-49fe-8393-a141c583e244 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f2f0682f3d34de2b9ae5f4371550ff4'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:45.293034 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-32c672d3-7bc8-49fe-8393-a141c583e244 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f2f0682f3d34de2b9ae5f4371550ff4'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:45.301048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-32c672d3-7bc8-49fe-8393-a141c583e244 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f2f0682f3d34de2b9ae5f4371550ff4'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:45.301667 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-32c672d3-7bc8-49fe-8393-a141c583e244 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f2f0682f3d34de2b9ae5f4371550ff4'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:45.308917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-32c672d3-7bc8-49fe-8393-a141c583e244 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f2f0682f3d34de2b9ae5f4371550ff4'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:45.314729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 811/1602] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:45 2022] DELETE /identity/v3/users/1f2f0682f3d34de2b9ae5f4371550ff4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:45.319914 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aef58-cf99-431e-8267-24b08a30a3cb None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:45.320423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aef58-cf99-431e-8267-24b08a30a3cb None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:45.320523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aef58-cf99-431e-8267-24b08a30a3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:45.340833 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-d51aef58-cf99-431e-8267-24b08a30a3cb None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:45.342018 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d51aef58-cf99-431e-8267-24b08a30a3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:45.389942 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:45.402548 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 792/1603] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:45.412496 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c672d3-7bc8-49fe-8393-a141c583e244 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:45.413935 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-32c672d3-7bc8-49fe-8393-a141c583e244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:45.477758 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c672d3-7bc8-49fe-8393-a141c583e244 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:45.482006 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b3f2f-d947-4b37-9cd5-3d0bf0a35c11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:45.482645 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b3f2f-d947-4b37-9cd5-3d0bf0a35c11 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:45.482908 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b3f2f-d947-4b37-9cd5-3d0bf0a35c11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:45.483202 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b3f2f-d947-4b37-9cd5-3d0bf0a35c11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:45.494663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b3f2f-d947-4b37-9cd5-3d0bf0a35c11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:45.496143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b3f2f-d947-4b37-9cd5-3d0bf0a35c11 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:45.507256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 812/1604] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:21:45.794176 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51aef58-cf99-431e-8267-24b08a30a3cb None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:45.795947 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d51aef58-cf99-431e-8267-24b08a30a3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:45.831022 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51aef58-cf99-431e-8267-24b08a30a3cb None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:45.850393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c9bb5-34ba-49fc-a512-63d0af8196c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:45.852491 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c9bb5-34ba-49fc-a512-63d0af8196c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:45.855255 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c9bb5-34ba-49fc-a512-63d0af8196c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:45.856895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c9bb5-34ba-49fc-a512-63d0af8196c1 admin admin] PATH_INFO: `/v3/projects/7ffd679007cd4fc8ab55ddf9cf6ca94f` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:45.859974 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c9bb5-34ba-49fc-a512-63d0af8196c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ffd679007cd4fc8ab55ddf9cf6ca94f)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:45.868297 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c9bb5-34ba-49fc-a512-63d0af8196c1 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:45.989553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-041c9bb5-34ba-49fc-a512-63d0af8196c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ffd679007cd4fc8ab55ddf9cf6ca94f', 'request_id': 'req-041c9bb5-34ba-49fc-a512-63d0af8196c1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.002189 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-041c9bb5-34ba-49fc-a512-63d0af8196c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ffd679007cd4fc8ab55ddf9cf6ca94f', 'request_id': 'req-041c9bb5-34ba-49fc-a512-63d0af8196c1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.013247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-041c9bb5-34ba-49fc-a512-63d0af8196c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ffd679007cd4fc8ab55ddf9cf6ca94f', 'request_id': 'req-041c9bb5-34ba-49fc-a512-63d0af8196c1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.034692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-041c9bb5-34ba-49fc-a512-63d0af8196c1 admin admin] The token cache is being invalidate because project 7ffd679007cd4fc8ab55ddf9cf6ca94f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:46.035169 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-041c9bb5-34ba-49fc-a512-63d0af8196c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.039510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-041c9bb5-34ba-49fc-a512-63d0af8196c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.042487 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 793/1605] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:45 2022] DELETE /identity/v3/projects/7ffd679007cd4fc8ab55ddf9cf6ca94f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:46.046302 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b3f2f-d947-4b37-9cd5-3d0bf0a35c11 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:46.061305 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b3f2f-d947-4b37-9cd5-3d0bf0a35c11 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:46.160378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24944b3a-05be-4c62-b5d9-aa07e99434c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:46.161354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24944b3a-05be-4c62-b5d9-aa07e99434c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:46.161572 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24944b3a-05be-4c62-b5d9-aa07e99434c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:46.161973 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24944b3a-05be-4c62-b5d9-aa07e99434c1 admin admin] PATH_INFO: `/v3/users/37ee2943603348c8a65f136e588e2421` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:46.166811 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24944b3a-05be-4c62-b5d9-aa07e99434c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37ee2943603348c8a65f136e588e2421)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:46.170303 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24944b3a-05be-4c62-b5d9-aa07e99434c1 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:46.260495 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-24944b3a-05be-4c62-b5d9-aa07e99434c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37ee2943603348c8a65f136e588e2421'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.283467 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-24944b3a-05be-4c62-b5d9-aa07e99434c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37ee2943603348c8a65f136e588e2421'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.288584 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:46.293678 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-24944b3a-05be-4c62-b5d9-aa07e99434c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37ee2943603348c8a65f136e588e2421'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.294620 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-24944b3a-05be-4c62-b5d9-aa07e99434c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37ee2943603348c8a65f136e588e2421'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.308667 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-24944b3a-05be-4c62-b5d9-aa07e99434c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37ee2943603348c8a65f136e588e2421'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.322500 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 813/1606] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:46 2022] DELETE /identity/v3/users/37ee2943603348c8a65f136e588e2421 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:46.409199 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c9bb5-34ba-49fc-a512-63d0af8196c1 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:46.411999 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-041c9bb5-34ba-49fc-a512-63d0af8196c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:46.426788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c9bb5-34ba-49fc-a512-63d0af8196c1 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:46.505317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3744eee-dd78-491a-bab3-ee816f498561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:46.506384 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3744eee-dd78-491a-bab3-ee816f498561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:46.506696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3744eee-dd78-491a-bab3-ee816f498561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:46.507160 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3744eee-dd78-491a-bab3-ee816f498561 admin admin] PATH_INFO: `/v3/users/b09e458a860940afb60a00483e88b2aa` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:46.509359 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3744eee-dd78-491a-bab3-ee816f498561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b09e458a860940afb60a00483e88b2aa)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:46.511984 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3744eee-dd78-491a-bab3-ee816f498561 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:46.581580 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e3744eee-dd78-491a-bab3-ee816f498561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b09e458a860940afb60a00483e88b2aa'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.582286 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e3744eee-dd78-491a-bab3-ee816f498561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b09e458a860940afb60a00483e88b2aa'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.595278 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e3744eee-dd78-491a-bab3-ee816f498561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b09e458a860940afb60a00483e88b2aa'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.606489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e3744eee-dd78-491a-bab3-ee816f498561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b09e458a860940afb60a00483e88b2aa'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.606489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e3744eee-dd78-491a-bab3-ee816f498561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b09e458a860940afb60a00483e88b2aa'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.615154 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:46.623317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 794/1607] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:46 2022] DELETE /identity/v3/users/b09e458a860940afb60a00483e88b2aa => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:46.624993 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77a9f3-f8c1-4858-b0b7-86e4d1c95d40 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:46.625276 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77a9f3-f8c1-4858-b0b7-86e4d1c95d40 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:46.625543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77a9f3-f8c1-4858-b0b7-86e4d1c95d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:46.663066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-ed77a9f3-f8c1-4858-b0b7-86e4d1c95d40 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:46.664351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ed77a9f3-f8c1-4858-b0b7-86e4d1c95d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:46.745099 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 814/1608] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:46.774794 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3744eee-dd78-491a-bab3-ee816f498561 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:46.777073 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-e3744eee-dd78-491a-bab3-ee816f498561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:46.837789 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3744eee-dd78-491a-bab3-ee816f498561 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:46.842453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5b610-2dea-4bc8-a25f-c4f65865a872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:46.843075 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5b610-2dea-4bc8-a25f-c4f65865a872 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:46.843343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5b610-2dea-4bc8-a25f-c4f65865a872 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:46.843642 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5b610-2dea-4bc8-a25f-c4f65865a872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:46.854525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5b610-2dea-4bc8-a25f-c4f65865a872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:46.855545 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5b610-2dea-4bc8-a25f-c4f65865a872 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:46.866583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 795/1609] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:46.883790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77a9f3-f8c1-4858-b0b7-86e4d1c95d40 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:46.896050 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ed77a9f3-f8c1-4858-b0b7-86e4d1c95d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:46.909862 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77a9f3-f8c1-4858-b0b7-86e4d1c95d40 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:46.916989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc51caf-2964-4431-b97f-5cdebd453a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:46.917635 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc51caf-2964-4431-b97f-5cdebd453a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:46.918072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc51caf-2964-4431-b97f-5cdebd453a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:46.918369 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc51caf-2964-4431-b97f-5cdebd453a7b admin admin] PATH_INFO: `/v3/projects/2ec803836f124dc7b18c834dafd23599` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:46.919943 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc51caf-2964-4431-b97f-5cdebd453a7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ec803836f124dc7b18c834dafd23599)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:46.922360 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc51caf-2964-4431-b97f-5cdebd453a7b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:46.970557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-efc51caf-2964-4431-b97f-5cdebd453a7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ec803836f124dc7b18c834dafd23599', 'request_id': 'req-efc51caf-2964-4431-b97f-5cdebd453a7b'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.976921 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-efc51caf-2964-4431-b97f-5cdebd453a7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ec803836f124dc7b18c834dafd23599', 'request_id': 'req-efc51caf-2964-4431-b97f-5cdebd453a7b'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.985156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-efc51caf-2964-4431-b97f-5cdebd453a7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ec803836f124dc7b18c834dafd23599', 'request_id': 'req-efc51caf-2964-4431-b97f-5cdebd453a7b'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.997220 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-efc51caf-2964-4431-b97f-5cdebd453a7b admin admin] The token cache is being invalidate because project 2ec803836f124dc7b18c834dafd23599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:46.997480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-efc51caf-2964-4431-b97f-5cdebd453a7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.998147 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-efc51caf-2964-4431-b97f-5cdebd453a7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:46.999883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 815/1610] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:46 2022] DELETE /identity/v3/projects/2ec803836f124dc7b18c834dafd23599 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:47.004810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5b610-2dea-4bc8-a25f-c4f65865a872 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:47.017952 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5b610-2dea-4bc8-a25f-c4f65865a872 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:47.029330 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc51caf-2964-4431-b97f-5cdebd453a7b admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:47.031516 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-efc51caf-2964-4431-b97f-5cdebd453a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:47.086677 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94724ce-215f-47b4-ba41-0f39a66562d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:47.087798 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94724ce-215f-47b4-ba41-0f39a66562d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:47.088457 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94724ce-215f-47b4-ba41-0f39a66562d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:47.088774 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94724ce-215f-47b4-ba41-0f39a66562d1 admin admin] PATH_INFO: `/v3/users/05214c00a99b495c9652073b75447c16` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:47.090165 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94724ce-215f-47b4-ba41-0f39a66562d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05214c00a99b495c9652073b75447c16)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:47.092216 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94724ce-215f-47b4-ba41-0f39a66562d1 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:47.114018 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc51caf-2964-4431-b97f-5cdebd453a7b admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:47.122712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0f38d-a5ad-420b-b611-73a2f34f6eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:47.123431 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0f38d-a5ad-420b-b611-73a2f34f6eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:47.123767 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0f38d-a5ad-420b-b611-73a2f34f6eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:47.124119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0f38d-a5ad-420b-b611-73a2f34f6eb3 admin admin] PATH_INFO: `/v3/users/c071160af585481dadb46225509f1520` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:47.125730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f0f38d-a5ad-420b-b611-73a2f34f6eb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c071160af585481dadb46225509f1520)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:47.127823 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f0f38d-a5ad-420b-b611-73a2f34f6eb3 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:47.134509 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:47.159072 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c94724ce-215f-47b4-ba41-0f39a66562d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05214c00a99b495c9652073b75447c16'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.159865 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c94724ce-215f-47b4-ba41-0f39a66562d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05214c00a99b495c9652073b75447c16'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.176173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c94724ce-215f-47b4-ba41-0f39a66562d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05214c00a99b495c9652073b75447c16'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.188217 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c94724ce-215f-47b4-ba41-0f39a66562d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05214c00a99b495c9652073b75447c16'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.190430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c94724ce-215f-47b4-ba41-0f39a66562d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05214c00a99b495c9652073b75447c16'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.207286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-64f0f38d-a5ad-420b-b611-73a2f34f6eb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c071160af585481dadb46225509f1520'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.217017 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 796/1611] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:47 2022] DELETE /identity/v3/users/05214c00a99b495c9652073b75447c16 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:47.219999 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94724ce-215f-47b4-ba41-0f39a66562d1 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:47.222339 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c94724ce-215f-47b4-ba41-0f39a66562d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:47.248481 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-64f0f38d-a5ad-420b-b611-73a2f34f6eb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c071160af585481dadb46225509f1520'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.268784 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-64f0f38d-a5ad-420b-b611-73a2f34f6eb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c071160af585481dadb46225509f1520'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.270131 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-64f0f38d-a5ad-420b-b611-73a2f34f6eb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c071160af585481dadb46225509f1520'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.277184 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:47.281989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-64f0f38d-a5ad-420b-b611-73a2f34f6eb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c071160af585481dadb46225509f1520'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.295020 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 816/1612] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:47 2022] DELETE /identity/v3/users/c071160af585481dadb46225509f1520 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:47.299254 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259ddfe-a782-4fdf-a8c0-657d63d9d0a3 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:47.301665 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259ddfe-a782-4fdf-a8c0-657d63d9d0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:47.302360 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259ddfe-a782-4fdf-a8c0-657d63d9d0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:47.334416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-0259ddfe-a782-4fdf-a8c0-657d63d9d0a3 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:47.335415 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94724ce-215f-47b4-ba41-0f39a66562d1 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:47.336199 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0259ddfe-a782-4fdf-a8c0-657d63d9d0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:47.406166 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f9d65-0118-475d-8a96-9bd26c9688e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:47.406847 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f9d65-0118-475d-8a96-9bd26c9688e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:47.407163 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f9d65-0118-475d-8a96-9bd26c9688e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:47.407525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f9d65-0118-475d-8a96-9bd26c9688e1 admin admin] PATH_INFO: `/v3/projects/882c8513a8124f01bc3ce7c42f4441a9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:47.408692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f9d65-0118-475d-8a96-9bd26c9688e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=882c8513a8124f01bc3ce7c42f4441a9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:47.411101 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f9d65-0118-475d-8a96-9bd26c9688e1 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:47.436347 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 817/1613] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:47.442477 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259ddfe-a782-4fdf-a8c0-657d63d9d0a3 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:47.444506 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0259ddfe-a782-4fdf-a8c0-657d63d9d0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:47.494272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ac1f9d65-0118-475d-8a96-9bd26c9688e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '882c8513a8124f01bc3ce7c42f4441a9', 'request_id': 'req-ac1f9d65-0118-475d-8a96-9bd26c9688e1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.509070 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ac1f9d65-0118-475d-8a96-9bd26c9688e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '882c8513a8124f01bc3ce7c42f4441a9', 'request_id': 'req-ac1f9d65-0118-475d-8a96-9bd26c9688e1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.517416 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ac1f9d65-0118-475d-8a96-9bd26c9688e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '882c8513a8124f01bc3ce7c42f4441a9', 'request_id': 'req-ac1f9d65-0118-475d-8a96-9bd26c9688e1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.539334 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259ddfe-a782-4fdf-a8c0-657d63d9d0a3 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:47.541387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ac1f9d65-0118-475d-8a96-9bd26c9688e1 admin admin] The token cache is being invalidate because project 882c8513a8124f01bc3ce7c42f4441a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:47.541889 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ac1f9d65-0118-475d-8a96-9bd26c9688e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.542629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ac1f9d65-0118-475d-8a96-9bd26c9688e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.544379 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 797/1614] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:47 2022] DELETE /identity/v3/projects/882c8513a8124f01bc3ce7c42f4441a9 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:47.547427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f9d65-0118-475d-8a96-9bd26c9688e1 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:47.547427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ac1f9d65-0118-475d-8a96-9bd26c9688e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:47.603349 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:47.623915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c6b6b-c948-47f0-a80b-d058148a2a26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:47.624568 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c6b6b-c948-47f0-a80b-d058148a2a26 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:47.624957 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c6b6b-c948-47f0-a80b-d058148a2a26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:47.625226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c6b6b-c948-47f0-a80b-d058148a2a26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:47.627075 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-107c6b6b-c948-47f0-a80b-d058148a2a26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:47.641200 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:47.645277 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f9d65-0118-475d-8a96-9bd26c9688e1 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:47.653467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90add48d-69d2-4ab4-8cfe-2c4b78c81f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:47.654288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90add48d-69d2-4ab4-8cfe-2c4b78c81f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:47.654649 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90add48d-69d2-4ab4-8cfe-2c4b78c81f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:47.655120 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90add48d-69d2-4ab4-8cfe-2c4b78c81f9f admin admin] PATH_INFO: `/v3/projects/a3788324495443d59834c9aa6b484bbc` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:47.656647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90add48d-69d2-4ab4-8cfe-2c4b78c81f9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3788324495443d59834c9aa6b484bbc)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:47.658625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90add48d-69d2-4ab4-8cfe-2c4b78c81f9f admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:47.702063 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c6b6b-c948-47f0-a80b-d058148a2a26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:47.703681 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c6b6b-c948-47f0-a80b-d058148a2a26 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:47.714356 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 818/1615] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:47.715234 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c6b6b-c948-47f0-a80b-d058148a2a26 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:47.716757 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-107c6b6b-c948-47f0-a80b-d058148a2a26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:47.720374 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-90add48d-69d2-4ab4-8cfe-2c4b78c81f9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3788324495443d59834c9aa6b484bbc', 'request_id': 'req-90add48d-69d2-4ab4-8cfe-2c4b78c81f9f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.729493 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c6b6b-c948-47f0-a80b-d058148a2a26 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:47.748263 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-90add48d-69d2-4ab4-8cfe-2c4b78c81f9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3788324495443d59834c9aa6b484bbc', 'request_id': 'req-90add48d-69d2-4ab4-8cfe-2c4b78c81f9f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.763970 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-90add48d-69d2-4ab4-8cfe-2c4b78c81f9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3788324495443d59834c9aa6b484bbc', 'request_id': 'req-90add48d-69d2-4ab4-8cfe-2c4b78c81f9f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.791772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-90add48d-69d2-4ab4-8cfe-2c4b78c81f9f admin admin] The token cache is being invalidate because project a3788324495443d59834c9aa6b484bbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:47.792124 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-90add48d-69d2-4ab4-8cfe-2c4b78c81f9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.794616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-90add48d-69d2-4ab4-8cfe-2c4b78c81f9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.800570 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 798/1616] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:47 2022] DELETE /identity/v3/projects/a3788324495443d59834c9aa6b484bbc => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:47.807423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a660f49-7fdf-447b-b8f7-498c6a4e9b51 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:47.807855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a660f49-7fdf-447b-b8f7-498c6a4e9b51 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:47.808136 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a660f49-7fdf-447b-b8f7-498c6a4e9b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:47.825887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b57112-d699-4f61-8be5-edfc0457708c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:47.826891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b57112-d699-4f61-8be5-edfc0457708c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:47.827312 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b57112-d699-4f61-8be5-edfc0457708c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:47.827753 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b57112-d699-4f61-8be5-edfc0457708c admin admin] PATH_INFO: `/v3/users/6f12af30bf3c45d2b467627ab0b5ad39` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:47.829095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b57112-d699-4f61-8be5-edfc0457708c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f12af30bf3c45d2b467627ab0b5ad39)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:47.832176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b57112-d699-4f61-8be5-edfc0457708c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:47.836046 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-9a660f49-7fdf-447b-b8f7-498c6a4e9b51 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:47.836046 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9a660f49-7fdf-447b-b8f7-498c6a4e9b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:47.844496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 799/1617] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:47.853410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1ae75-b59b-4aad-93cc-f6de4d6359ab None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:47.853721 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1ae75-b59b-4aad-93cc-f6de4d6359ab None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:47.853994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1ae75-b59b-4aad-93cc-f6de4d6359ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:47.883712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-84c1ae75-b59b-4aad-93cc-f6de4d6359ab None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:47.885064 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-84c1ae75-b59b-4aad-93cc-f6de4d6359ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:47.900586 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 800/1618] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:47.900941 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-22b57112-d699-4f61-8be5-edfc0457708c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f12af30bf3c45d2b467627ab0b5ad39'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.905820 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1ae75-b59b-4aad-93cc-f6de4d6359ab None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:47.916580 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-22b57112-d699-4f61-8be5-edfc0457708c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f12af30bf3c45d2b467627ab0b5ad39'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.917593 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1ae75-b59b-4aad-93cc-f6de4d6359ab None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:47.923658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970e8b0-8910-4f99-bd46-6e706c7f8017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:47.924210 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970e8b0-8910-4f99-bd46-6e706c7f8017 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:47.925620 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970e8b0-8910-4f99-bd46-6e706c7f8017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:47.925977 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970e8b0-8910-4f99-bd46-6e706c7f8017 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:47.926463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-22b57112-d699-4f61-8be5-edfc0457708c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f12af30bf3c45d2b467627ab0b5ad39'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.927043 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970e8b0-8910-4f99-bd46-6e706c7f8017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:21:47.927160 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-22b57112-d699-4f61-8be5-edfc0457708c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f12af30bf3c45d2b467627ab0b5ad39'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.927581 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970e8b0-8910-4f99-bd46-6e706c7f8017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:47.930294 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970e8b0-8910-4f99-bd46-6e706c7f8017 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:47.935028 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-22b57112-d699-4f61-8be5-edfc0457708c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f12af30bf3c45d2b467627ab0b5ad39'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:47.944128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 819/1619] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:47 2022] DELETE /identity/v3/users/6f12af30bf3c45d2b467627ab0b5ad39 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:47.955364 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b57112-d699-4f61-8be5-edfc0457708c admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:47.955929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 801/1620] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:21:47.960702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-22b57112-d699-4f61-8be5-edfc0457708c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:47.977489 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:48.044657 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b57112-d699-4f61-8be5-edfc0457708c admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:48.051627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bcdc6-7893-434d-9898-851ae39f97a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:48.052353 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bcdc6-7893-434d-9898-851ae39f97a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:48.052939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bcdc6-7893-434d-9898-851ae39f97a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:48.053340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bcdc6-7893-434d-9898-851ae39f97a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:48.054138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1bcdc6-7893-434d-9898-851ae39f97a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:48.056417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1bcdc6-7893-434d-9898-851ae39f97a2 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:48.085315 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 820/1621] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:47 2022] POST /identity/v3/projects => generated 401 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:21:48.089807 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970e8b0-8910-4f99-bd46-6e706c7f8017 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:48.100580 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970e8b0-8910-4f99-bd46-6e706c7f8017 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:48.162199 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8324619-5b7c-4e41-a90a-ea4568dc0369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:48.162823 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8324619-5b7c-4e41-a90a-ea4568dc0369 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:48.163098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8324619-5b7c-4e41-a90a-ea4568dc0369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:48.163366 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8324619-5b7c-4e41-a90a-ea4568dc0369 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:48.164034 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8324619-5b7c-4e41-a90a-ea4568dc0369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:48.165970 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8324619-5b7c-4e41-a90a-ea4568dc0369 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:48.190487 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 802/1622] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:48 2022] POST /identity/v3/users => generated 301 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:48.195015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bcdc6-7893-434d-9898-851ae39f97a2 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:48.207626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bcdc6-7893-434d-9898-851ae39f97a2 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:48.212521 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6dfb78-6c1d-4908-997b-08e9fa04f45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:48.213147 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6dfb78-6c1d-4908-997b-08e9fa04f45d admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:48.213465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6dfb78-6c1d-4908-997b-08e9fa04f45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:48.213777 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6dfb78-6c1d-4908-997b-08e9fa04f45d admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:48.214495 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6dfb78-6c1d-4908-997b-08e9fa04f45d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:48.216285 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6dfb78-6c1d-4908-997b-08e9fa04f45d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:48.231637 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 821/1623] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:48.234798 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8324619-5b7c-4e41-a90a-ea4568dc0369 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:48.246619 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8324619-5b7c-4e41-a90a-ea4568dc0369 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:48.253493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f92d4f-9276-4023-96f9-267a688ef2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:48.253897 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f92d4f-9276-4023-96f9-267a688ef2b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:48.254365 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f92d4f-9276-4023-96f9-267a688ef2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:48.254835 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f92d4f-9276-4023-96f9-267a688ef2b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:48.255512 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f92d4f-9276-4023-96f9-267a688ef2b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:48.257433 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f92d4f-9276-4023-96f9-267a688ef2b7 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:48.273308 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 803/1624] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:48.274504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6dfb78-6c1d-4908-997b-08e9fa04f45d admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:48.288223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6dfb78-6c1d-4908-997b-08e9fa04f45d admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:48.294898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee95c2-869b-47c3-a5ee-5ba4f95a9590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:48.295512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee95c2-869b-47c3-a5ee-5ba4f95a9590 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:48.295825 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee95c2-869b-47c3-a5ee-5ba4f95a9590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:48.296187 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee95c2-869b-47c3-a5ee-5ba4f95a9590 admin admin] PATH_INFO: `/v3/projects/f5e0aff1b242422ca2278c4c247fabdc/users/e8141ba651484d6584ef06df5292f4b0/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:48.314199 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee95c2-869b-47c3-a5ee-5ba4f95a9590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5e0aff1b242422ca2278c4c247fabdc, user_id=e8141ba651484d6584ef06df5292f4b0, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:48.317195 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee95c2-869b-47c3-a5ee-5ba4f95a9590 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:48.339789 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 822/1625] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:48 2022] PUT /identity/v3/projects/f5e0aff1b242422ca2278c4c247fabdc/users/e8141ba651484d6584ef06df5292f4b0/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:48.353879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa9fb6-eba1-453a-aa23-f802bb4e1c70 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:48.353879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa9fb6-eba1-453a-aa23-f802bb4e1c70 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:48.354191 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa9fb6-eba1-453a-aa23-f802bb4e1c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:48.392166 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-2eaa9fb6-eba1-453a-aa23-f802bb4e1c70 None None] MFA Rules not processed for user `e8141ba651484d6584ef06df5292f4b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:48.394087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2eaa9fb6-eba1-453a-aa23-f802bb4e1c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:48.433732 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:48.525255 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 804/1626] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:48 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:48.530238 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee95c2-869b-47c3-a5ee-5ba4f95a9590 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:48.530583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa9fb6-eba1-453a-aa23-f802bb4e1c70 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:48.532131 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d2ee95c2-869b-47c3-a5ee-5ba4f95a9590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:48.618691 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee95c2-869b-47c3-a5ee-5ba4f95a9590 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:48.623490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa9fb6-eba1-453a-aa23-f802bb4e1c70 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:48.628268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:48.629373 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:48.629638 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:48.630002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac admin admin] PATH_INFO: `/v3/projects/97aa42626a4b42c58f69fc92f6b63a49` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:48.630325 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4423b8-894f-45fb-87e2-f798300113e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:48.630946 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4423b8-894f-45fb-87e2-f798300113e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:48.631230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4423b8-894f-45fb-87e2-f798300113e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:48.631331 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac admin admin] RBAC: Authorizing `identity:delete_project(project_id=97aa42626a4b42c58f69fc92f6b63a49)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:48.631504 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4423b8-894f-45fb-87e2-f798300113e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:48.632177 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4423b8-894f-45fb-87e2-f798300113e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:48.633421 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:48.634182 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4423b8-894f-45fb-87e2-f798300113e6 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:48.679196 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 805/1627] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:48 2022] POST /identity/v3/projects => generated 397 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:48.679837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4423b8-894f-45fb-87e2-f798300113e6 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:48.681692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ec4423b8-894f-45fb-87e2-f798300113e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:48.707057 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97aa42626a4b42c58f69fc92f6b63a49', 'request_id': 'req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:48.719757 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97aa42626a4b42c58f69fc92f6b63a49', 'request_id': 'req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:48.732599 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97aa42626a4b42c58f69fc92f6b63a49', 'request_id': 'req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:48.735397 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:48.749907 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac admin admin] The token cache is being invalidate because project 97aa42626a4b42c58f69fc92f6b63a49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:48.750288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:48.752372 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:48.755029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 823/1628] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:48 2022] DELETE /identity/v3/projects/97aa42626a4b42c58f69fc92f6b63a49 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:48.755994 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:48.757771 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:48.777957 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4423b8-894f-45fb-87e2-f798300113e6 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:48.845698 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7fbfe-c9fb-46e8-8b51-89cfcf425dac admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:48.860728 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796be592-77c1-4aef-8902-7bfd0f082ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:48.861423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796be592-77c1-4aef-8902-7bfd0f082ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:48.861794 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796be592-77c1-4aef-8902-7bfd0f082ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:48.862091 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796be592-77c1-4aef-8902-7bfd0f082ee2 admin admin] PATH_INFO: `/v3/projects/ffb97b7c0032411aa3038d53ad5a0b5c` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:48.862988 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c8a56-741b-4467-9fbd-0ebcc815451e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:48.863232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796be592-77c1-4aef-8902-7bfd0f082ee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffb97b7c0032411aa3038d53ad5a0b5c)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:48.863636 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c8a56-741b-4467-9fbd-0ebcc815451e admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:48.863919 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c8a56-741b-4467-9fbd-0ebcc815451e admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:48.864192 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c8a56-741b-4467-9fbd-0ebcc815451e admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:48.864962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c8a56-741b-4467-9fbd-0ebcc815451e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:48.865180 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796be592-77c1-4aef-8902-7bfd0f082ee2 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:48.866958 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c8a56-741b-4467-9fbd-0ebcc815451e admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:48.867754 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:48.890752 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 824/1629] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:48 2022] POST /identity/v3/users => generated 305 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:48.892040 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e47e6-af02-44f7-a0a4-3df9ac79d7ea None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:48.892327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e47e6-af02-44f7-a0a4-3df9ac79d7ea None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:48.892600 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e47e6-af02-44f7-a0a4-3df9ac79d7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:48.914656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-382e47e6-af02-44f7-a0a4-3df9ac79d7ea None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:48.915911 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-382e47e6-af02-44f7-a0a4-3df9ac79d7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:48.922787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-796be592-77c1-4aef-8902-7bfd0f082ee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffb97b7c0032411aa3038d53ad5a0b5c', 'request_id': 'req-796be592-77c1-4aef-8902-7bfd0f082ee2'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:48.931285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-796be592-77c1-4aef-8902-7bfd0f082ee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffb97b7c0032411aa3038d53ad5a0b5c', 'request_id': 'req-796be592-77c1-4aef-8902-7bfd0f082ee2'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:48.937747 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-796be592-77c1-4aef-8902-7bfd0f082ee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffb97b7c0032411aa3038d53ad5a0b5c', 'request_id': 'req-796be592-77c1-4aef-8902-7bfd0f082ee2'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:48.951523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-796be592-77c1-4aef-8902-7bfd0f082ee2 admin admin] The token cache is being invalidate because project ffb97b7c0032411aa3038d53ad5a0b5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:48.951839 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-796be592-77c1-4aef-8902-7bfd0f082ee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:48.952510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-796be592-77c1-4aef-8902-7bfd0f082ee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:48.954068 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 806/1630] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:48 2022] DELETE /identity/v3/projects/ffb97b7c0032411aa3038d53ad5a0b5c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:48.957302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796be592-77c1-4aef-8902-7bfd0f082ee2 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:48.957302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-796be592-77c1-4aef-8902-7bfd0f082ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:48.994089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 825/1631] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:48.994951 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e47e6-af02-44f7-a0a4-3df9ac79d7ea None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:49.004776 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-382e47e6-af02-44f7-a0a4-3df9ac79d7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:49.016960 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e47e6-af02-44f7-a0a4-3df9ac79d7ea None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:49.022337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c688d4f-9923-4686-903e-cefb02b63866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:49.023297 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c688d4f-9923-4686-903e-cefb02b63866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.023804 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c688d4f-9923-4686-903e-cefb02b63866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.024157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c688d4f-9923-4686-903e-cefb02b63866 admin admin] PATH_INFO: `/v3/users/a11cb163eb8d495080a918bfba220a8f` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.025351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c688d4f-9923-4686-903e-cefb02b63866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a11cb163eb8d495080a918bfba220a8f)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:49.027364 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c688d4f-9923-4686-903e-cefb02b63866 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:49.035023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796be592-77c1-4aef-8902-7bfd0f082ee2 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:49.039486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb10ae-9c2f-48d5-b2b7-f22fb7670522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:49.040180 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb10ae-9c2f-48d5-b2b7-f22fb7670522 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.040389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb10ae-9c2f-48d5-b2b7-f22fb7670522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.040651 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb10ae-9c2f-48d5-b2b7-f22fb7670522 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.041279 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb10ae-9c2f-48d5-b2b7-f22fb7670522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:49.042832 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb10ae-9c2f-48d5-b2b7-f22fb7670522 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:49.054852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 807/1632] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:49.059232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dc753-7349-443c-b36e-a1e8fef40981 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.059475 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dc753-7349-443c-b36e-a1e8fef40981 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.059776 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dc753-7349-443c-b36e-a1e8fef40981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.080002 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-f42dc753-7349-443c-b36e-a1e8fef40981 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:49.081936 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-f42dc753-7349-443c-b36e-a1e8fef40981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:49.087813 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-8c688d4f-9923-4686-903e-cefb02b63866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a11cb163eb8d495080a918bfba220a8f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:49.090939 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 808/1633] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:49.091991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42dc753-7349-443c-b36e-a1e8fef40981 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:49.102376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42dc753-7349-443c-b36e-a1e8fef40981 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:49.108779 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4cb586-e25d-4b56-acb6-da8a955b146c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:49.109604 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-8c688d4f-9923-4686-903e-cefb02b63866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a11cb163eb8d495080a918bfba220a8f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:49.109932 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cb586-e25d-4b56-acb6-da8a955b146c admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.110413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cb586-e25d-4b56-acb6-da8a955b146c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.110888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cb586-e25d-4b56-acb6-da8a955b146c admin admin] PATH_INFO: `/v3/projects/57da666f45ea4cffa2008bacf257bd93/users/2099aa2409ec4165b92b70e30dc99a3a/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.121755 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-8c688d4f-9923-4686-903e-cefb02b63866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a11cb163eb8d495080a918bfba220a8f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:49.122481 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-8c688d4f-9923-4686-903e-cefb02b63866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a11cb163eb8d495080a918bfba220a8f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:49.128830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4cb586-e25d-4b56-acb6-da8a955b146c admin admin] RBAC: Authorizing `identity:create_grant(project_id=57da666f45ea4cffa2008bacf257bd93, user_id=2099aa2409ec4165b92b70e30dc99a3a, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:49.132492 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4cb586-e25d-4b56-acb6-da8a955b146c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:49.139117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:49.149062 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-8c688d4f-9923-4686-903e-cefb02b63866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a11cb163eb8d495080a918bfba220a8f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:49.152815 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 809/1634] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:49 2022] PUT /identity/v3/projects/57da666f45ea4cffa2008bacf257bd93/users/2099aa2409ec4165b92b70e30dc99a3a/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:49.157297 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4cb586-e25d-4b56-acb6-da8a955b146c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:49.158674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 826/1635] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:48 2022] DELETE /identity/v3/users/a11cb163eb8d495080a918bfba220a8f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:49.159658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ec4cb586-e25d-4b56-acb6-da8a955b146c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:49.159904 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c688d4f-9923-4686-903e-cefb02b63866 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:49.161401 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8c688d4f-9923-4686-903e-cefb02b63866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:49.237075 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:49.245855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c688d4f-9923-4686-903e-cefb02b63866 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:49.248548 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4cb586-e25d-4b56-acb6-da8a955b146c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:49.252932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ada05-763b-42fd-861d-04fee74f5e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:49.254275 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9a45c-d66a-405b-a212-a79176a0234e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:49.254691 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ada05-763b-42fd-861d-04fee74f5e85 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.254900 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9a45c-d66a-405b-a212-a79176a0234e admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.255190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9a45c-d66a-405b-a212-a79176a0234e admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.255466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9a45c-d66a-405b-a212-a79176a0234e admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.255805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ada05-763b-42fd-861d-04fee74f5e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.256194 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9a45c-d66a-405b-a212-a79176a0234e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:21:49.256479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ada05-763b-42fd-861d-04fee74f5e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.256579 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9a45c-d66a-405b-a212-a79176a0234e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:49.258427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ada05-763b-42fd-861d-04fee74f5e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:49.258549 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9a45c-d66a-405b-a212-a79176a0234e admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:49.261670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ada05-763b-42fd-861d-04fee74f5e85 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:49.272074 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 810/1636] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:21:49.276997 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 827/1637] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:49.277527 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9a45c-d66a-405b-a212-a79176a0234e admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:49.281192 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ada05-763b-42fd-861d-04fee74f5e85 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:49.292369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9a45c-d66a-405b-a212-a79176a0234e admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:49.295699 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ada05-763b-42fd-861d-04fee74f5e85 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:49.298124 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de4540-9f7d-4173-99b5-bb231582b919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:49.299036 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de4540-9f7d-4173-99b5-bb231582b919 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.299398 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de4540-9f7d-4173-99b5-bb231582b919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.299744 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de4540-9f7d-4173-99b5-bb231582b919 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.300430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de4540-9f7d-4173-99b5-bb231582b919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:49.302628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de4540-9f7d-4173-99b5-bb231582b919 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:49.304902 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199feea-7f2c-47ba-9ee2-ce39f6c7f315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:49.305919 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199feea-7f2c-47ba-9ee2-ce39f6c7f315 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.306784 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199feea-7f2c-47ba-9ee2-ce39f6c7f315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.307257 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199feea-7f2c-47ba-9ee2-ce39f6c7f315 admin admin] PATH_INFO: `/v3/projects/57da666f45ea4cffa2008bacf257bd93/users/2099aa2409ec4165b92b70e30dc99a3a/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.310880 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199feea-7f2c-47ba-9ee2-ce39f6c7f315 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57da666f45ea4cffa2008bacf257bd93, user_id=2099aa2409ec4165b92b70e30dc99a3a, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:49.314245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199feea-7f2c-47ba-9ee2-ce39f6c7f315 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:49.339754 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 811/1638] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:49 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:49.341241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de4540-9f7d-4173-99b5-bb231582b919 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:49.345043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 828/1639] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:49 2022] PUT /identity/v3/projects/57da666f45ea4cffa2008bacf257bd93/users/2099aa2409ec4165b92b70e30dc99a3a/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:49.356231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfd34b-4e14-4392-8b14-bdbf2598c64e None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.356717 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfd34b-4e14-4392-8b14-bdbf2598c64e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.356906 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfd34b-4e14-4392-8b14-bdbf2598c64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.389316 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-6bcfd34b-4e14-4392-8b14-bdbf2598c64e None None] MFA Rules not processed for user `2099aa2409ec4165b92b70e30dc99a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:49.390583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-6bcfd34b-4e14-4392-8b14-bdbf2598c64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:49.417083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de4540-9f7d-4173-99b5-bb231582b919 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:49.441827 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f9ff7-42c7-4326-a85c-ef815c0e77f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:49.442595 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f9ff7-42c7-4326-a85c-ef815c0e77f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.442877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f9ff7-42c7-4326-a85c-ef815c0e77f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.443147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f9ff7-42c7-4326-a85c-ef815c0e77f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.443814 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f9ff7-42c7-4326-a85c-ef815c0e77f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:49.451004 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f9ff7-42c7-4326-a85c-ef815c0e77f9 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:49.492570 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 812/1640] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:49 2022] POST /identity/v3/users => generated 283 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:49.497257 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f9ff7-42c7-4326-a85c-ef815c0e77f9 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:49.501606 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 829/1641] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:49 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:49.514141 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfd34b-4e14-4392-8b14-bdbf2598c64e None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:49.516112 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-6bcfd34b-4e14-4392-8b14-bdbf2598c64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:49.518703 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f9ff7-42c7-4326-a85c-ef815c0e77f9 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:49.525296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d145f-80f3-405c-947e-04e5e0af9ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:49.525918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d145f-80f3-405c-947e-04e5e0af9ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.526230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d145f-80f3-405c-947e-04e5e0af9ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.526744 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d145f-80f3-405c-947e-04e5e0af9ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.528056 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d145f-80f3-405c-947e-04e5e0af9ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:49.529669 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d145f-80f3-405c-947e-04e5e0af9ee0 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:49.543189 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 813/1642] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:49.548976 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d145f-80f3-405c-947e-04e5e0af9ee0 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:49.566422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d145f-80f3-405c-947e-04e5e0af9ee0 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:49.575291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457231c-0b63-44de-bd7a-bb8eafcd1323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:49.576013 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457231c-0b63-44de-bd7a-bb8eafcd1323 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.576292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457231c-0b63-44de-bd7a-bb8eafcd1323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.576760 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457231c-0b63-44de-bd7a-bb8eafcd1323 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.577441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457231c-0b63-44de-bd7a-bb8eafcd1323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:49.579069 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457231c-0b63-44de-bd7a-bb8eafcd1323 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:49.586432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfd34b-4e14-4392-8b14-bdbf2598c64e None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:49.590335 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cbf2b1-745f-4d26-b853-afc5d224e602 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:49.590967 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbf2b1-745f-4d26-b853-afc5d224e602 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.591216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbf2b1-745f-4d26-b853-afc5d224e602 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.591535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbf2b1-745f-4d26-b853-afc5d224e602 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.605202 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 814/1643] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:49.605202 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457231c-0b63-44de-bd7a-bb8eafcd1323 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:49.605450 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cbf2b1-745f-4d26-b853-afc5d224e602 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:49.606701 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cbf2b1-745f-4d26-b853-afc5d224e602 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:49.617024 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3457231c-0b63-44de-bd7a-bb8eafcd1323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:49.619388 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 830/1644] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:49 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:21:49.620515 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cbf2b1-745f-4d26-b853-afc5d224e602 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:49.628401 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:49.635400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457231c-0b63-44de-bd7a-bb8eafcd1323 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:49.635855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cbf2b1-745f-4d26-b853-afc5d224e602 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:49.643988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d30139-8462-4111-8fda-7fc80d54b11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:49.644804 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d30139-8462-4111-8fda-7fc80d54b11f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.645082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d30139-8462-4111-8fda-7fc80d54b11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.645349 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d30139-8462-4111-8fda-7fc80d54b11f admin admin] PATH_INFO: `/v3/projects/49bae1df6cf54e24bb4615b115ddb23a` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.646479 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d30139-8462-4111-8fda-7fc80d54b11f admin admin] RBAC: Authorizing `identity:delete_project(project_id=49bae1df6cf54e24bb4615b115ddb23a)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:49.648668 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d30139-8462-4111-8fda-7fc80d54b11f admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:49.650835 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bd790-0cda-49f5-ae03-916d03424014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:49.651847 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bd790-0cda-49f5-ae03-916d03424014 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.652287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bd790-0cda-49f5-ae03-916d03424014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.652579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bd790-0cda-49f5-ae03-916d03424014 admin admin] PATH_INFO: `/v3/projects/9ab6184daf7148fa88092d98631e1f0e/users/17555a4bdd2e424aad4fc9b5ff38d505/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.673212 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614bd790-0cda-49f5-ae03-916d03424014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ab6184daf7148fa88092d98631e1f0e, user_id=17555a4bdd2e424aad4fc9b5ff38d505, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:49.676121 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614bd790-0cda-49f5-ae03-916d03424014 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:49.698767 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 831/1645] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:49 2022] PUT /identity/v3/projects/9ab6184daf7148fa88092d98631e1f0e/users/17555a4bdd2e424aad4fc9b5ff38d505/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:49.703807 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a2d30139-8462-4111-8fda-7fc80d54b11f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49bae1df6cf54e24bb4615b115ddb23a', 'request_id': 'req-a2d30139-8462-4111-8fda-7fc80d54b11f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:49.710284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25c45d-e6bc-4eaa-8160-7b7ccf71aee4 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.710635 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25c45d-e6bc-4eaa-8160-7b7ccf71aee4 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.711003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25c45d-e6bc-4eaa-8160-7b7ccf71aee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.713629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a2d30139-8462-4111-8fda-7fc80d54b11f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49bae1df6cf54e24bb4615b115ddb23a', 'request_id': 'req-a2d30139-8462-4111-8fda-7fc80d54b11f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:49.720073 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a2d30139-8462-4111-8fda-7fc80d54b11f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49bae1df6cf54e24bb4615b115ddb23a', 'request_id': 'req-a2d30139-8462-4111-8fda-7fc80d54b11f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:49.750523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a2d30139-8462-4111-8fda-7fc80d54b11f admin admin] The token cache is being invalidate because project 49bae1df6cf54e24bb4615b115ddb23a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:49.750703 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a2d30139-8462-4111-8fda-7fc80d54b11f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:49.752259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a2d30139-8462-4111-8fda-7fc80d54b11f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:49.755691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 815/1646] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:49 2022] DELETE /identity/v3/projects/49bae1df6cf54e24bb4615b115ddb23a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:49.757600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d30139-8462-4111-8fda-7fc80d54b11f admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:49.759317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a2d30139-8462-4111-8fda-7fc80d54b11f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:49.767198 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-ab25c45d-e6bc-4eaa-8160-7b7ccf71aee4 None None] MFA Rules not processed for user `17555a4bdd2e424aad4fc9b5ff38d505`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:49.769485 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ab25c45d-e6bc-4eaa-8160-7b7ccf71aee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:49.790072 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:49.838532 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d30139-8462-4111-8fda-7fc80d54b11f admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:49.842237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2263f7-0c02-449d-8e39-2ab9dcc0dceb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:49.843417 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2263f7-0c02-449d-8e39-2ab9dcc0dceb service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.843790 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2263f7-0c02-449d-8e39-2ab9dcc0dceb service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.844275 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2263f7-0c02-449d-8e39-2ab9dcc0dceb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.846454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-0c2263f7-0c02-449d-8e39-2ab9dcc0dceb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:49.892666 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 832/1647] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:49.894686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7536862-e088-4314-baef-bf28e259be4a None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:49.894943 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7536862-e088-4314-baef-bf28e259be4a None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:49.895217 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7536862-e088-4314-baef-bf28e259be4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:49.935926 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-d7536862-e088-4314-baef-bf28e259be4a None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:49.946180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d7536862-e088-4314-baef-bf28e259be4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:50.016591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2263f7-0c02-449d-8e39-2ab9dcc0dceb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:50.019600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2263f7-0c02-449d-8e39-2ab9dcc0dceb service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:50.032138 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 816/1648] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:49 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:50.101820 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 833/1649] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:50.109992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c858c-4636-425f-acd5-3b46b4c580c0 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:50.110536 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c858c-4636-425f-acd5-3b46b4c580c0 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:50.111118 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c858c-4636-425f-acd5-3b46b4c580c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:50.153695 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-523c858c-4636-425f-acd5-3b46b4c580c0 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:50.154956 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-523c858c-4636-425f-acd5-3b46b4c580c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:50.173614 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 817/1650] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:50.174379 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7536862-e088-4314-baef-bf28e259be4a None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:50.193872 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7536862-e088-4314-baef-bf28e259be4a None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:50.200060 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc6024-3fa1-4fe2-9998-5cafc7f060b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:50.200947 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc6024-3fa1-4fe2-9998-5cafc7f060b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:50.201136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc6024-3fa1-4fe2-9998-5cafc7f060b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:50.201382 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc6024-3fa1-4fe2-9998-5cafc7f060b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:50.202141 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc6024-3fa1-4fe2-9998-5cafc7f060b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:21:50.202770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc6024-3fa1-4fe2-9998-5cafc7f060b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:50.205320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc6024-3fa1-4fe2-9998-5cafc7f060b7 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:50.224777 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 834/1651] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:21:50.229492 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c858c-4636-425f-acd5-3b46b4c580c0 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:50.250999 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c858c-4636-425f-acd5-3b46b4c580c0 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:50.265047 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89722b4a-b02e-4592-8a84-c4a5f14ba506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:50.265929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89722b4a-b02e-4592-8a84-c4a5f14ba506 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:50.266655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89722b4a-b02e-4592-8a84-c4a5f14ba506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:50.266655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89722b4a-b02e-4592-8a84-c4a5f14ba506 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:50.267096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89722b4a-b02e-4592-8a84-c4a5f14ba506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:50.269102 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89722b4a-b02e-4592-8a84-c4a5f14ba506 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:50.306568 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 818/1652] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:50 2022] POST /identity/v3/projects => generated 375 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:50.310685 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc6024-3fa1-4fe2-9998-5cafc7f060b7 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:50.333658 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc6024-3fa1-4fe2-9998-5cafc7f060b7 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:50.401398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:50.413722 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00dd0d-974b-4a3b-904a-ee25bf7a93e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:50.414900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00dd0d-974b-4a3b-904a-ee25bf7a93e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:50.415283 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00dd0d-974b-4a3b-904a-ee25bf7a93e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:50.416058 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00dd0d-974b-4a3b-904a-ee25bf7a93e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:50.416904 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00dd0d-974b-4a3b-904a-ee25bf7a93e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:50.420036 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00dd0d-974b-4a3b-904a-ee25bf7a93e7 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:50.457253 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89722b4a-b02e-4592-8a84-c4a5f14ba506 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:50.481140 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 835/1653] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:50 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:50.485524 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00dd0d-974b-4a3b-904a-ee25bf7a93e7 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:50.496928 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00dd0d-974b-4a3b-904a-ee25bf7a93e7 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:50.502179 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c72872-9a93-418f-a14a-957f4088d37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:50.503620 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c72872-9a93-418f-a14a-957f4088d37d admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:50.503979 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c72872-9a93-418f-a14a-957f4088d37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:50.504322 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c72872-9a93-418f-a14a-957f4088d37d admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:50.505349 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c72872-9a93-418f-a14a-957f4088d37d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:50.507064 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c72872-9a93-418f-a14a-957f4088d37d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:50.538196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 836/1654] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:50.539198 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c72872-9a93-418f-a14a-957f4088d37d admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:50.564793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c72872-9a93-418f-a14a-957f4088d37d admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:50.564973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89722b4a-b02e-4592-8a84-c4a5f14ba506 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:50.572045 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fa77f-8b75-41cd-87f3-3e0f68102a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:50.572714 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fa77f-8b75-41cd-87f3-3e0f68102a95 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:50.573045 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fa77f-8b75-41cd-87f3-3e0f68102a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:50.573417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fa77f-8b75-41cd-87f3-3e0f68102a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:50.574063 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fa77f-8b75-41cd-87f3-3e0f68102a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:50.575981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fa77f-8b75-41cd-87f3-3e0f68102a95 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:50.580234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8eea83-430e-429a-9a2c-44010d1b7a9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:50.580947 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8eea83-430e-429a-9a2c-44010d1b7a9b service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:50.581203 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8eea83-430e-429a-9a2c-44010d1b7a9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:50.581481 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8eea83-430e-429a-9a2c-44010d1b7a9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:50.626235 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 837/1655] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:50.627607 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fa77f-8b75-41cd-87f3-3e0f68102a95 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:50.652199 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fa77f-8b75-41cd-87f3-3e0f68102a95 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:50.657627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff6986-eddd-4fb8-ab99-9c4dab29aebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:50.658832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff6986-eddd-4fb8-ab99-9c4dab29aebc admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:50.659726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff6986-eddd-4fb8-ab99-9c4dab29aebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:50.660330 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff6986-eddd-4fb8-ab99-9c4dab29aebc admin admin] PATH_INFO: `/v3/projects/94a62d7750aa4ac0b67f58b1cd3edfd0/users/93fb2924916a452782bb9b32091c5807/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:50.681362 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8eea83-430e-429a-9a2c-44010d1b7a9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:50.682767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8eea83-430e-429a-9a2c-44010d1b7a9b service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:50.699126 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff6986-eddd-4fb8-ab99-9c4dab29aebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=94a62d7750aa4ac0b67f58b1cd3edfd0, user_id=93fb2924916a452782bb9b32091c5807, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:50.700165 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:50.702152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff6986-eddd-4fb8-ab99-9c4dab29aebc admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:50.708060 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 819/1656] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:21:50.719413 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 838/1657] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:50 2022] PUT /identity/v3/projects/94a62d7750aa4ac0b67f58b1cd3edfd0/users/93fb2924916a452782bb9b32091c5807/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:50.732163 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3179a-fd9b-4464-a2e8-d744baec8986 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:50.732449 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3179a-fd9b-4464-a2e8-d744baec8986 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:50.732725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3179a-fd9b-4464-a2e8-d744baec8986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:50.763784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-bbd3179a-fd9b-4464-a2e8-d744baec8986 None None] MFA Rules not processed for user `93fb2924916a452782bb9b32091c5807`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:50.765113 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-bbd3179a-fd9b-4464-a2e8-d744baec8986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:50.842897 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 820/1658] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:50.856536 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff6986-eddd-4fb8-ab99-9c4dab29aebc admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:50.858436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b3ff6986-eddd-4fb8-ab99-9c4dab29aebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:50.921677 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff6986-eddd-4fb8-ab99-9c4dab29aebc admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:50.926231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627842c-dac9-4d12-a971-a06d19c000a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:50.926845 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627842c-dac9-4d12-a971-a06d19c000a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:50.927162 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627842c-dac9-4d12-a971-a06d19c000a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:50.927402 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627842c-dac9-4d12-a971-a06d19c000a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:50.937123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627842c-dac9-4d12-a971-a06d19c000a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:50.938887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627842c-dac9-4d12-a971-a06d19c000a1 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:50.960867 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 839/1659] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:54.473653 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3179a-fd9b-4464-a2e8-d744baec8986 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:54.475197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-bbd3179a-fd9b-4464-a2e8-d744baec8986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:54.548069 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3179a-fd9b-4464-a2e8-d744baec8986 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:54.554068 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e856d-0890-4459-814d-d5ba6a3f0bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:54.555786 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e856d-0890-4459-814d-d5ba6a3f0bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:54.556071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e856d-0890-4459-814d-d5ba6a3f0bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:54.556343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e856d-0890-4459-814d-d5ba6a3f0bd3 admin admin] PATH_INFO: `/v3/users/e8141ba651484d6584ef06df5292f4b0` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:54.559122 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e856d-0890-4459-814d-d5ba6a3f0bd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8141ba651484d6584ef06df5292f4b0)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:54.561253 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e856d-0890-4459-814d-d5ba6a3f0bd3 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:54.623927 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0b6e856d-0890-4459-814d-d5ba6a3f0bd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8141ba651484d6584ef06df5292f4b0'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:54.624810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0b6e856d-0890-4459-814d-d5ba6a3f0bd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8141ba651484d6584ef06df5292f4b0'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:54.638041 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0b6e856d-0890-4459-814d-d5ba6a3f0bd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8141ba651484d6584ef06df5292f4b0'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:54.646739 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0b6e856d-0890-4459-814d-d5ba6a3f0bd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8141ba651484d6584ef06df5292f4b0'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:54.647599 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0b6e856d-0890-4459-814d-d5ba6a3f0bd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8141ba651484d6584ef06df5292f4b0'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:54.661484 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 821/1660] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:54 2022] DELETE /identity/v3/users/e8141ba651484d6584ef06df5292f4b0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:54.665817 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7e47c-6c68-4279-8434-7b63b35f1458 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:54.666291 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7e47c-6c68-4279-8434-7b63b35f1458 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:54.666405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7e47c-6c68-4279-8434-7b63b35f1458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:54.686915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-bdd7e47c-6c68-4279-8434-7b63b35f1458 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:54.688143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-bdd7e47c-6c68-4279-8434-7b63b35f1458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:54.864599 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 840/1661] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:54.874151 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e856d-0890-4459-814d-d5ba6a3f0bd3 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:54.875554 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-0b6e856d-0890-4459-814d-d5ba6a3f0bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:54.887684 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:54.935348 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e856d-0890-4459-814d-d5ba6a3f0bd3 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:54.940703 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e785d5-bcf2-4d9d-904e-4d91310366a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:54.941318 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e785d5-bcf2-4d9d-904e-4d91310366a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:54.941597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e785d5-bcf2-4d9d-904e-4d91310366a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:54.941864 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e785d5-bcf2-4d9d-904e-4d91310366a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:54.956453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e785d5-bcf2-4d9d-904e-4d91310366a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:54.957892 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e785d5-bcf2-4d9d-904e-4d91310366a7 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:54.967889 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 822/1662] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:55.269957 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7e47c-6c68-4279-8434-7b63b35f1458 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:55.271476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-bdd7e47c-6c68-4279-8434-7b63b35f1458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:55.291081 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7e47c-6c68-4279-8434-7b63b35f1458 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:55.297175 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442873cd-7ebc-4cdd-9804-85118958dff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:55.297936 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442873cd-7ebc-4cdd-9804-85118958dff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:55.299338 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442873cd-7ebc-4cdd-9804-85118958dff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:55.300263 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442873cd-7ebc-4cdd-9804-85118958dff5 admin admin] PATH_INFO: `/v3/users/93fb2924916a452782bb9b32091c5807` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:55.301655 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442873cd-7ebc-4cdd-9804-85118958dff5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93fb2924916a452782bb9b32091c5807)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:55.304182 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442873cd-7ebc-4cdd-9804-85118958dff5 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:55.354136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:55.363146 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-442873cd-7ebc-4cdd-9804-85118958dff5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93fb2924916a452782bb9b32091c5807'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:55.380824 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-442873cd-7ebc-4cdd-9804-85118958dff5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93fb2924916a452782bb9b32091c5807'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:55.389456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-442873cd-7ebc-4cdd-9804-85118958dff5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93fb2924916a452782bb9b32091c5807'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:55.390067 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-442873cd-7ebc-4cdd-9804-85118958dff5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93fb2924916a452782bb9b32091c5807'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:55.402449 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-442873cd-7ebc-4cdd-9804-85118958dff5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93fb2924916a452782bb9b32091c5807'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:55.409172 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 841/1663] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:55 2022] DELETE /identity/v3/users/93fb2924916a452782bb9b32091c5807 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:55.413725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d987a3f-841d-4200-8e90-9d6440f99d9e None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:55.414009 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d987a3f-841d-4200-8e90-9d6440f99d9e None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:55.414306 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d987a3f-841d-4200-8e90-9d6440f99d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:55.436352 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-4d987a3f-841d-4200-8e90-9d6440f99d9e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:55.437691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-4d987a3f-841d-4200-8e90-9d6440f99d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:55.516970 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 823/1664] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:55.527711 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442873cd-7ebc-4cdd-9804-85118958dff5 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:55.529215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-442873cd-7ebc-4cdd-9804-85118958dff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:55.553683 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d987a3f-841d-4200-8e90-9d6440f99d9e None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:55.555348 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-4d987a3f-841d-4200-8e90-9d6440f99d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:55.569421 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d987a3f-841d-4200-8e90-9d6440f99d9e None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:55.574409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:55.575106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:55.575391 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:55.575712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1 admin admin] PATH_INFO: `/v3/projects/f5e0aff1b242422ca2278c4c247fabdc` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:55.576956 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5e0aff1b242422ca2278c4c247fabdc)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:55.579305 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:55.603024 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442873cd-7ebc-4cdd-9804-85118958dff5 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:55.606704 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e0c7a-fe87-4039-a945-d652e0aeb5fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:55.607312 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e0c7a-fe87-4039-a945-d652e0aeb5fe service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:55.607671 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e0c7a-fe87-4039-a945-d652e0aeb5fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:55.607961 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e0c7a-fe87-4039-a945-d652e0aeb5fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:55.618100 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e0c7a-fe87-4039-a945-d652e0aeb5fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:55.619471 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e0c7a-fe87-4039-a945-d652e0aeb5fe service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:55.627758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5e0aff1b242422ca2278c4c247fabdc', 'request_id': 'req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:55.629022 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 842/1665] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:55.635461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5e0aff1b242422ca2278c4c247fabdc', 'request_id': 'req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:55.641379 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5e0aff1b242422ca2278c4c247fabdc', 'request_id': 'req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:55.656116 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1 admin admin] The token cache is being invalidate because project f5e0aff1b242422ca2278c4c247fabdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:55.656386 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:55.657045 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:55.658658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 824/1666] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:55 2022] DELETE /identity/v3/projects/f5e0aff1b242422ca2278c4c247fabdc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:55.663066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e0c7a-fe87-4039-a945-d652e0aeb5fe service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:55.672990 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e0c7a-fe87-4039-a945-d652e0aeb5fe service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:55.748767 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c96fe0-874b-411e-941e-b72dbc9698e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:55.749787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c96fe0-874b-411e-941e-b72dbc9698e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:55.750109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c96fe0-874b-411e-941e-b72dbc9698e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:55.750423 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c96fe0-874b-411e-941e-b72dbc9698e0 admin admin] PATH_INFO: `/v3/users/2099aa2409ec4165b92b70e30dc99a3a` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:55.751708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c96fe0-874b-411e-941e-b72dbc9698e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2099aa2409ec4165b92b70e30dc99a3a)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:55.753874 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c96fe0-874b-411e-941e-b72dbc9698e0 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:55.811544 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c4c96fe0-874b-411e-941e-b72dbc9698e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2099aa2409ec4165b92b70e30dc99a3a'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:55.823383 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:55.825975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c4c96fe0-874b-411e-941e-b72dbc9698e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2099aa2409ec4165b92b70e30dc99a3a'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:55.838642 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c4c96fe0-874b-411e-941e-b72dbc9698e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2099aa2409ec4165b92b70e30dc99a3a'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:55.839947 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c4c96fe0-874b-411e-941e-b72dbc9698e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2099aa2409ec4165b92b70e30dc99a3a'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:55.849882 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c4c96fe0-874b-411e-941e-b72dbc9698e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2099aa2409ec4165b92b70e30dc99a3a'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:55.857038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 843/1667] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:55 2022] DELETE /identity/v3/users/2099aa2409ec4165b92b70e30dc99a3a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:55.941514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:55.943460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:55.956485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedf539-6dfc-440d-b9e5-d7b87911fdc1 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:55.994566 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:56.037456 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8041f8-42de-469e-9081-c3b26cc2bcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:56.038111 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8041f8-42de-469e-9081-c3b26cc2bcac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:56.038429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8041f8-42de-469e-9081-c3b26cc2bcac admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:56.038652 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8041f8-42de-469e-9081-c3b26cc2bcac admin admin] PATH_INFO: `/v3/projects/94a62d7750aa4ac0b67f58b1cd3edfd0` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:56.040207 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8041f8-42de-469e-9081-c3b26cc2bcac admin admin] RBAC: Authorizing `identity:delete_project(project_id=94a62d7750aa4ac0b67f58b1cd3edfd0)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:56.042162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8041f8-42de-469e-9081-c3b26cc2bcac admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:56.093145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bf8041f8-42de-469e-9081-c3b26cc2bcac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94a62d7750aa4ac0b67f58b1cd3edfd0', 'request_id': 'req-bf8041f8-42de-469e-9081-c3b26cc2bcac'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:56.100966 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bf8041f8-42de-469e-9081-c3b26cc2bcac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94a62d7750aa4ac0b67f58b1cd3edfd0', 'request_id': 'req-bf8041f8-42de-469e-9081-c3b26cc2bcac'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:56.106977 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bf8041f8-42de-469e-9081-c3b26cc2bcac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94a62d7750aa4ac0b67f58b1cd3edfd0', 'request_id': 'req-bf8041f8-42de-469e-9081-c3b26cc2bcac'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:56.119049 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bf8041f8-42de-469e-9081-c3b26cc2bcac admin admin] The token cache is being invalidate because project 94a62d7750aa4ac0b67f58b1cd3edfd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:56.119338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bf8041f8-42de-469e-9081-c3b26cc2bcac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:56.120005 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bf8041f8-42de-469e-9081-c3b26cc2bcac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:56.121508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 825/1668] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:55 2022] DELETE /identity/v3/projects/94a62d7750aa4ac0b67f58b1cd3edfd0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:56.126453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13298723-007d-405f-b02e-5f44440e3520 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:56.126683 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13298723-007d-405f-b02e-5f44440e3520 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:56.126963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13298723-007d-405f-b02e-5f44440e3520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:56.146499 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-13298723-007d-405f-b02e-5f44440e3520 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:56.147968 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-13298723-007d-405f-b02e-5f44440e3520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:56.168646 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8041f8-42de-469e-9081-c3b26cc2bcac admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:56.170053 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-bf8041f8-42de-469e-9081-c3b26cc2bcac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:56.231739 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 844/1669] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:56.242354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b83c2-4ca9-4b38-9610-bea182afbaba None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:56.242641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b83c2-4ca9-4b38-9610-bea182afbaba None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:56.242918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b83c2-4ca9-4b38-9610-bea182afbaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:56.274250 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-f91b83c2-4ca9-4b38-9610-bea182afbaba None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:56.275983 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f91b83c2-4ca9-4b38-9610-bea182afbaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:56.286381 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8041f8-42de-469e-9081-c3b26cc2bcac admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:56.287493 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 845/1670] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:56.291684 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b83c2-4ca9-4b38-9610-bea182afbaba None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:56.295391 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673a246-b8fc-4890-b21f-2a907c3f0cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:56.296279 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673a246-b8fc-4890-b21f-2a907c3f0cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:56.296568 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673a246-b8fc-4890-b21f-2a907c3f0cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:56.296845 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673a246-b8fc-4890-b21f-2a907c3f0cd3 admin admin] PATH_INFO: `/v3/projects/57da666f45ea4cffa2008bacf257bd93` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:56.298128 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673a246-b8fc-4890-b21f-2a907c3f0cd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57da666f45ea4cffa2008bacf257bd93)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:56.300801 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673a246-b8fc-4890-b21f-2a907c3f0cd3 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:56.306570 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b83c2-4ca9-4b38-9610-bea182afbaba None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:56.316641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22504248-67e3-4acd-9bfd-1a601799109f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:56.317286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22504248-67e3-4acd-9bfd-1a601799109f admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:56.317653 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22504248-67e3-4acd-9bfd-1a601799109f admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:56.317926 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22504248-67e3-4acd-9bfd-1a601799109f admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:56.319398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22504248-67e3-4acd-9bfd-1a601799109f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:21:56.320242 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22504248-67e3-4acd-9bfd-1a601799109f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:56.323019 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22504248-67e3-4acd-9bfd-1a601799109f admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:56.340094 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 846/1671] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:21:56.345798 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22504248-67e3-4acd-9bfd-1a601799109f admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:56.361607 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22504248-67e3-4acd-9bfd-1a601799109f admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:56.365508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5673a246-b8fc-4890-b21f-2a907c3f0cd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57da666f45ea4cffa2008bacf257bd93', 'request_id': 'req-5673a246-b8fc-4890-b21f-2a907c3f0cd3'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:56.375948 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5673a246-b8fc-4890-b21f-2a907c3f0cd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57da666f45ea4cffa2008bacf257bd93', 'request_id': 'req-5673a246-b8fc-4890-b21f-2a907c3f0cd3'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:56.400167 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5673a246-b8fc-4890-b21f-2a907c3f0cd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57da666f45ea4cffa2008bacf257bd93', 'request_id': 'req-5673a246-b8fc-4890-b21f-2a907c3f0cd3'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:56.414916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:56.441296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5673a246-b8fc-4890-b21f-2a907c3f0cd3 admin admin] The token cache is being invalidate because project 57da666f45ea4cffa2008bacf257bd93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:56.441855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5673a246-b8fc-4890-b21f-2a907c3f0cd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:56.444426 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5673a246-b8fc-4890-b21f-2a907c3f0cd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:56.453760 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 826/1672] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:56 2022] DELETE /identity/v3/projects/57da666f45ea4cffa2008bacf257bd93 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:56.458689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a47666-aaa2-4449-a865-0e4d2f9aa8de None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:56.459836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a47666-aaa2-4449-a865-0e4d2f9aa8de None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:56.460140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a47666-aaa2-4449-a865-0e4d2f9aa8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:56.497030 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-04a47666-aaa2-4449-a865-0e4d2f9aa8de None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:56.498590 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-04a47666-aaa2-4449-a865-0e4d2f9aa8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:56.563933 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d0bac-ddd1-4f38-975d-f3cc26b3c8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:56.564566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d0bac-ddd1-4f38-975d-f3cc26b3c8b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:56.564844 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d0bac-ddd1-4f38-975d-f3cc26b3c8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:56.565643 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d0bac-ddd1-4f38-975d-f3cc26b3c8b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:56.566395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d0bac-ddd1-4f38-975d-f3cc26b3c8b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:56.568361 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d0bac-ddd1-4f38-975d-f3cc26b3c8b6 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:56.625622 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 847/1673] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:56 2022] POST /identity/v3/projects => generated 369 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:56.630230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d0bac-ddd1-4f38-975d-f3cc26b3c8b6 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:56.632025 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-7e0d0bac-ddd1-4f38-975d-f3cc26b3c8b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:56.639995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 827/1674] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:56.648446 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d0c50-536e-4f55-a72a-1b1dc066c52f None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:56.649984 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d0c50-536e-4f55-a72a-1b1dc066c52f None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:56.650686 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d0c50-536e-4f55-a72a-1b1dc066c52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:56.687279 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-a51d0c50-536e-4f55-a72a-1b1dc066c52f None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:56.689813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a51d0c50-536e-4f55-a72a-1b1dc066c52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:56.758359 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:56.787260 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d0bac-ddd1-4f38-975d-f3cc26b3c8b6 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:56.792632 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26944190-698b-41b5-9399-ef99022a71d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:56.793512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26944190-698b-41b5-9399-ef99022a71d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:56.793893 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26944190-698b-41b5-9399-ef99022a71d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:56.794176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26944190-698b-41b5-9399-ef99022a71d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:56.794817 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26944190-698b-41b5-9399-ef99022a71d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:56.797040 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26944190-698b-41b5-9399-ef99022a71d0 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:56.801640 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 828/1675] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:56.807875 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d0c50-536e-4f55-a72a-1b1dc066c52f None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:56.822726 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d0c50-536e-4f55-a72a-1b1dc066c52f None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:56.827899 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 848/1676] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:56 2022] POST /identity/v3/users => generated 285 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:21:56.833352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26944190-698b-41b5-9399-ef99022a71d0 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:56.835186 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea4be6-b902-4d85-99dc-a7773a34799d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:56.835929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea4be6-b902-4d85-99dc-a7773a34799d admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:56.836280 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea4be6-b902-4d85-99dc-a7773a34799d admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:56.836627 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea4be6-b902-4d85-99dc-a7773a34799d admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:56.837508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea4be6-b902-4d85-99dc-a7773a34799d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:21:56.837882 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea4be6-b902-4d85-99dc-a7773a34799d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:56.839961 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea4be6-b902-4d85-99dc-a7773a34799d admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:56.844463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26944190-698b-41b5-9399-ef99022a71d0 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:56.850275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e7536-15a6-49a0-9442-fa361e531adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:56.850961 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e7536-15a6-49a0-9442-fa361e531adc admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:56.851318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e7536-15a6-49a0-9442-fa361e531adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:56.852345 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e7536-15a6-49a0-9442-fa361e531adc admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:56.853254 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e7536-15a6-49a0-9442-fa361e531adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:56.855931 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e7536-15a6-49a0-9442-fa361e531adc admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:56.859519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 829/1677] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:21:56.860442 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea4be6-b902-4d85-99dc-a7773a34799d admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:56.873688 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea4be6-b902-4d85-99dc-a7773a34799d admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:56.874215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 849/1678] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:56.878538 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e7536-15a6-49a0-9442-fa361e531adc admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:56.889660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e7536-15a6-49a0-9442-fa361e531adc admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:56.893951 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87dbbf6-8248-4edd-983a-32f08ac4345c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:56.894861 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac95423-bc3e-4c28-93aa-bfa6047e6429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:56.895322 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dbbf6-8248-4edd-983a-32f08ac4345c admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:56.895474 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac95423-bc3e-4c28-93aa-bfa6047e6429 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:56.895847 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac95423-bc3e-4c28-93aa-bfa6047e6429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:56.896074 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dbbf6-8248-4edd-983a-32f08ac4345c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:56.896221 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac95423-bc3e-4c28-93aa-bfa6047e6429 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:56.897050 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dbbf6-8248-4edd-983a-32f08ac4345c admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:56.897615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac95423-bc3e-4c28-93aa-bfa6047e6429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:56.899519 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac95423-bc3e-4c28-93aa-bfa6047e6429 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:56.900161 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87dbbf6-8248-4edd-983a-32f08ac4345c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:56.902237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87dbbf6-8248-4edd-983a-32f08ac4345c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:56.915272 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 850/1679] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:21:56.916497 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac95423-bc3e-4c28-93aa-bfa6047e6429 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:56.931251 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac95423-bc3e-4c28-93aa-bfa6047e6429 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:56.942790 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 830/1680] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:56 2022] POST /identity/v3/projects => generated 383 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:56.945972 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6262c-9522-409b-a5a4-8fceef092288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:56.946713 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6262c-9522-409b-a5a4-8fceef092288 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:56.946841 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87dbbf6-8248-4edd-983a-32f08ac4345c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:56.946984 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6262c-9522-409b-a5a4-8fceef092288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:56.947368 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6262c-9522-409b-a5a4-8fceef092288 admin admin] PATH_INFO: `/v3/projects/76a154ba2f174dac88b6478f1eaf41d0/users/3ae046fff4dc4099856df86934b62ca5/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:56.978329 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6262c-9522-409b-a5a4-8fceef092288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76a154ba2f174dac88b6478f1eaf41d0, user_id=3ae046fff4dc4099856df86934b62ca5, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:56.981114 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6262c-9522-409b-a5a4-8fceef092288 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:56.998464 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 851/1681] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:56 2022] PUT /identity/v3/projects/76a154ba2f174dac88b6478f1eaf41d0/users/3ae046fff4dc4099856df86934b62ca5/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:57.010575 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa296ee4-3cb0-4037-8d52-e6c811de3b02 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:57.010973 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa296ee4-3cb0-4037-8d52-e6c811de3b02 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:57.011322 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa296ee4-3cb0-4037-8d52-e6c811de3b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:57.050711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87dbbf6-8248-4edd-983a-32f08ac4345c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:57.058790 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:57.072691 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-aa296ee4-3cb0-4037-8d52-e6c811de3b02 None None] MFA Rules not processed for user `3ae046fff4dc4099856df86934b62ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:57.074853 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-aa296ee4-3cb0-4037-8d52-e6c811de3b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:57.135472 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee1b0b-8d42-4f63-af3e-0d21bd50b121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:57.136122 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee1b0b-8d42-4f63-af3e-0d21bd50b121 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:57.136402 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee1b0b-8d42-4f63-af3e-0d21bd50b121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:57.136676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee1b0b-8d42-4f63-af3e-0d21bd50b121 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:57.137312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee1b0b-8d42-4f63-af3e-0d21bd50b121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:57.139223 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee1b0b-8d42-4f63-af3e-0d21bd50b121 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:57.170909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 831/1682] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:56 2022] POST /identity/v3/users => generated 292 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:21:57.172114 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee1b0b-8d42-4f63-af3e-0d21bd50b121 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:57.186398 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee1b0b-8d42-4f63-af3e-0d21bd50b121 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:57.192945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c969f-06db-4a69-9367-46b14eb32816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:57.193708 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c969f-06db-4a69-9367-46b14eb32816 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:57.193995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c969f-06db-4a69-9367-46b14eb32816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:57.194268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c969f-06db-4a69-9367-46b14eb32816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:57.194906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c969f-06db-4a69-9367-46b14eb32816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:57.197408 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c969f-06db-4a69-9367-46b14eb32816 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:57.203145 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 852/1683] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:57.211303 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 832/1684] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:57.214871 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa296ee4-3cb0-4037-8d52-e6c811de3b02 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:57.218642 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c969f-06db-4a69-9367-46b14eb32816 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:57.221107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-be0c969f-06db-4a69-9367-46b14eb32816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:57.225705 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa296ee4-3cb0-4037-8d52-e6c811de3b02 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:57.230311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30cfeb-c77e-41e3-8af6-594fe3f37aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:57.230917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30cfeb-c77e-41e3-8af6-594fe3f37aae admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:57.231190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30cfeb-c77e-41e3-8af6-594fe3f37aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:57.232116 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30cfeb-c77e-41e3-8af6-594fe3f37aae admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:57.232755 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30cfeb-c77e-41e3-8af6-594fe3f37aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:57.234342 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30cfeb-c77e-41e3-8af6-594fe3f37aae admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:57.248837 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 853/1685] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:57.255707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30cfeb-c77e-41e3-8af6-594fe3f37aae admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:57.271007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30cfeb-c77e-41e3-8af6-594fe3f37aae admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:57.276719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd719b-6238-406d-8143-847dbb848367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:57.277406 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd719b-6238-406d-8143-847dbb848367 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:57.277683 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd719b-6238-406d-8143-847dbb848367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:57.277980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd719b-6238-406d-8143-847dbb848367 admin admin] PATH_INFO: `/v3/projects/74f2cff2a1eb4545a3ed021d7b0e4166/users/0f0cd831346d453da1624b14bb4f3361/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:57.294607 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd719b-6238-406d-8143-847dbb848367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f2cff2a1eb4545a3ed021d7b0e4166, user_id=0f0cd831346d453da1624b14bb4f3361, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:57.297365 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd719b-6238-406d-8143-847dbb848367 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:57.317720 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 854/1686] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:57 2022] PUT /identity/v3/projects/74f2cff2a1eb4545a3ed021d7b0e4166/users/0f0cd831346d453da1624b14bb4f3361/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:57.328918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3876c-7ca6-48b5-88db-c7241cd77ec7 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:57.332515 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c969f-06db-4a69-9367-46b14eb32816 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:57.336466 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3876c-7ca6-48b5-88db-c7241cd77ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:57.336759 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3876c-7ca6-48b5-88db-c7241cd77ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:57.388529 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-80f3876c-7ca6-48b5-88db-c7241cd77ec7 None None] MFA Rules not processed for user `0f0cd831346d453da1624b14bb4f3361`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:57.391533 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-80f3876c-7ca6-48b5-88db-c7241cd77ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:57.410436 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794db5f-ca16-45fa-81c0-6678970f0347 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce5ec441b06408bbadd6fc040af0da5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:57.411487 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794db5f-ca16-45fa-81c0-6678970f0347 service glance] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:57.411990 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794db5f-ca16-45fa-81c0-6678970f0347 service glance] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:57.412458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794db5f-ca16-45fa-81c0-6678970f0347 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:57.442702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:57.466142 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:57.485966 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 855/1687] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:57.492838 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794db5f-ca16-45fa-81c0-6678970f0347 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:57.494791 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794db5f-ca16-45fa-81c0-6678970f0347 service glance] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:57.499020 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3876c-7ca6-48b5-88db-c7241cd77ec7 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:57.500841 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-80f3876c-7ca6-48b5-88db-c7241cd77ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:57.505156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 833/1688] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:57.564399 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3876c-7ca6-48b5-88db-c7241cd77ec7 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:57.569683 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f5a44-4695-4bdc-8ca4-73dfda11d809 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:57.570426 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f5a44-4695-4bdc-8ca4-73dfda11d809 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:57.570725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f5a44-4695-4bdc-8ca4-73dfda11d809 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:57.571063 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f5a44-4695-4bdc-8ca4-73dfda11d809 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:57.584818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f5a44-4695-4bdc-8ca4-73dfda11d809 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:57.586345 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f5a44-4695-4bdc-8ca4-73dfda11d809 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:57.597968 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 856/1689] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:21:57.620380 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794db5f-ca16-45fa-81c0-6678970f0347 service glance] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:57.706016 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794db5f-ca16-45fa-81c0-6678970f0347 service glance] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:57.711830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f948af9-7c97-45af-bdd1-bc5747d1d4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:57.712582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f948af9-7c97-45af-bdd1-bc5747d1d4db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:57.712937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f948af9-7c97-45af-bdd1-bc5747d1d4db admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:57.713264 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f948af9-7c97-45af-bdd1-bc5747d1d4db admin admin] PATH_INFO: `/v3/users/0f0cd831346d453da1624b14bb4f3361` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:57.714448 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f948af9-7c97-45af-bdd1-bc5747d1d4db admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f0cd831346d453da1624b14bb4f3361)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:57.716476 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f948af9-7c97-45af-bdd1-bc5747d1d4db admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:57.799491 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1f948af9-7c97-45af-bdd1-bc5747d1d4db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f0cd831346d453da1624b14bb4f3361'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:57.800215 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1f948af9-7c97-45af-bdd1-bc5747d1d4db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f0cd831346d453da1624b14bb4f3361'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:57.817269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1f948af9-7c97-45af-bdd1-bc5747d1d4db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f0cd831346d453da1624b14bb4f3361'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:57.827333 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1f948af9-7c97-45af-bdd1-bc5747d1d4db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f0cd831346d453da1624b14bb4f3361'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:57.828021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1f948af9-7c97-45af-bdd1-bc5747d1d4db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f0cd831346d453da1624b14bb4f3361'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:57.855511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 834/1690] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:21:57 2022] DELETE /identity/v3/users/0f0cd831346d453da1624b14bb4f3361 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:21:57.860084 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da46b0-4586-4378-adb6-b2dbacecf446 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:57.861593 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da46b0-4586-4378-adb6-b2dbacecf446 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:57.862121 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da46b0-4586-4378-adb6-b2dbacecf446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:57.887425 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-b2da46b0-4586-4378-adb6-b2dbacecf446 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:57.889822 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b2da46b0-4586-4378-adb6-b2dbacecf446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:57.981205 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 857/1691] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:57.992317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f948af9-7c97-45af-bdd1-bc5747d1d4db admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:57.995674 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1f948af9-7c97-45af-bdd1-bc5747d1d4db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:58.128910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f948af9-7c97-45af-bdd1-bc5747d1d4db admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:58.137862 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e174e75-4548-4d4c-b367-130842858ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:58.140041 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e174e75-4548-4d4c-b367-130842858ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:58.140227 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e174e75-4548-4d4c-b367-130842858ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:58.141706 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e174e75-4548-4d4c-b367-130842858ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:58.187053 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e174e75-4548-4d4c-b367-130842858ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:58.191783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e174e75-4548-4d4c-b367-130842858ec1 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:58.197764 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:58.212265 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 835/1692] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:21:58.823999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da46b0-4586-4378-adb6-b2dbacecf446 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:58.825464 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b2da46b0-4586-4378-adb6-b2dbacecf446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:58.838729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da46b0-4586-4378-adb6-b2dbacecf446 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:58.851685 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78d8f3-e9df-41a0-bca5-3247de7cc0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:58.852637 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78d8f3-e9df-41a0-bca5-3247de7cc0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:58.853109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78d8f3-e9df-41a0-bca5-3247de7cc0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:58.853465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78d8f3-e9df-41a0-bca5-3247de7cc0db admin admin] PATH_INFO: `/v3/projects/74f2cff2a1eb4545a3ed021d7b0e4166` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:58.854805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78d8f3-e9df-41a0-bca5-3247de7cc0db admin admin] RBAC: Authorizing `identity:delete_project(project_id=74f2cff2a1eb4545a3ed021d7b0e4166)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:58.856858 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78d8f3-e9df-41a0-bca5-3247de7cc0db admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:58.909977 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7b78d8f3-e9df-41a0-bca5-3247de7cc0db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74f2cff2a1eb4545a3ed021d7b0e4166', 'request_id': 'req-7b78d8f3-e9df-41a0-bca5-3247de7cc0db'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:58.917699 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7b78d8f3-e9df-41a0-bca5-3247de7cc0db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74f2cff2a1eb4545a3ed021d7b0e4166', 'request_id': 'req-7b78d8f3-e9df-41a0-bca5-3247de7cc0db'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:58.925746 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7b78d8f3-e9df-41a0-bca5-3247de7cc0db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74f2cff2a1eb4545a3ed021d7b0e4166', 'request_id': 'req-7b78d8f3-e9df-41a0-bca5-3247de7cc0db'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:58.932237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:58.937726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7b78d8f3-e9df-41a0-bca5-3247de7cc0db admin admin] The token cache is being invalidate because project 74f2cff2a1eb4545a3ed021d7b0e4166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:21:58.938406 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7b78d8f3-e9df-41a0-bca5-3247de7cc0db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:58.939468 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7b78d8f3-e9df-41a0-bca5-3247de7cc0db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:21:58.943744 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 858/1693] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:21:58 2022] DELETE /identity/v3/projects/74f2cff2a1eb4545a3ed021d7b0e4166 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:58.946444 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c179089-bab1-4247-ae5c-0c4b225f39aa None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:58.946894 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c179089-bab1-4247-ae5c-0c4b225f39aa None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:58.946970 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c179089-bab1-4247-ae5c-0c4b225f39aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:58.969263 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-2c179089-bab1-4247-ae5c-0c4b225f39aa None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:58.970606 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2c179089-bab1-4247-ae5c-0c4b225f39aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:59.043300 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 836/1694] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:21:59.052282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e01d6-73a1-4692-8185-41475cbf87c2 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:59.054345 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e01d6-73a1-4692-8185-41475cbf87c2 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:59.054345 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e01d6-73a1-4692-8185-41475cbf87c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:59.074268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-713e01d6-73a1-4692-8185-41475cbf87c2 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:59.075670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-713e01d6-73a1-4692-8185-41475cbf87c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:59.090172 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 859/1695] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:59.094554 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c179089-bab1-4247-ae5c-0c4b225f39aa None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:59.104963 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c179089-bab1-4247-ae5c-0c4b225f39aa None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:59.109847 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa9245-327a-4def-9a40-9433bd599912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:59.110468 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa9245-327a-4def-9a40-9433bd599912 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:59.110748 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa9245-327a-4def-9a40-9433bd599912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:59.111037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa9245-327a-4def-9a40-9433bd599912 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:59.111706 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa9245-327a-4def-9a40-9433bd599912 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:21:59.112661 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa9245-327a-4def-9a40-9433bd599912 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:59.113697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa9245-327a-4def-9a40-9433bd599912 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:59.127003 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 837/1696] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:21:59.132955 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e01d6-73a1-4692-8185-41475cbf87c2 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:59.143945 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e01d6-73a1-4692-8185-41475cbf87c2 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:59.151040 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3825ea1-cdcb-4d01-97b6-fa3b5a49fbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:59.152083 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3825ea1-cdcb-4d01-97b6-fa3b5a49fbca admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:59.152083 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3825ea1-cdcb-4d01-97b6-fa3b5a49fbca admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:59.152252 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3825ea1-cdcb-4d01-97b6-fa3b5a49fbca admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:59.152838 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3825ea1-cdcb-4d01-97b6-fa3b5a49fbca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:59.154791 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3825ea1-cdcb-4d01-97b6-fa3b5a49fbca admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:59.201172 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 860/1697] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:21:59 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:59.206353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa9245-327a-4def-9a40-9433bd599912 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:59.221310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa9245-327a-4def-9a40-9433bd599912 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:59.344453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463629d6-99ff-40ff-9926-d1be08b1e3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:59.345367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463629d6-99ff-40ff-9926-d1be08b1e3fe admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:59.345656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463629d6-99ff-40ff-9926-d1be08b1e3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:59.345932 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463629d6-99ff-40ff-9926-d1be08b1e3fe admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:59.347113 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463629d6-99ff-40ff-9926-d1be08b1e3fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:59.349189 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463629d6-99ff-40ff-9926-d1be08b1e3fe admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:59.399355 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 838/1698] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:21:59 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:21:59.403819 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3825ea1-cdcb-4d01-97b6-fa3b5a49fbca admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:59.417097 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3825ea1-cdcb-4d01-97b6-fa3b5a49fbca admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:59.422512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80e482-32ae-49b7-9b76-549b8b68d051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:59.422797 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80e482-32ae-49b7-9b76-549b8b68d051 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:59.423081 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80e482-32ae-49b7-9b76-549b8b68d051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:59.423356 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80e482-32ae-49b7-9b76-549b8b68d051 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:59.424056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80e482-32ae-49b7-9b76-549b8b68d051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:59.426222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80e482-32ae-49b7-9b76-549b8b68d051 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:59.451687 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 861/1699] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:59.452892 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463629d6-99ff-40ff-9926-d1be08b1e3fe admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:59.469132 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463629d6-99ff-40ff-9926-d1be08b1e3fe admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:59.475678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42340f6b-7449-49ec-a2d4-dfb743b49c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:59.478861 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42340f6b-7449-49ec-a2d4-dfb743b49c6d admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:59.479415 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42340f6b-7449-49ec-a2d4-dfb743b49c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:59.479897 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42340f6b-7449-49ec-a2d4-dfb743b49c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:59.481421 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42340f6b-7449-49ec-a2d4-dfb743b49c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:59.484579 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42340f6b-7449-49ec-a2d4-dfb743b49c6d admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:59.505459 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 839/1700] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:21:59.509276 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80e482-32ae-49b7-9b76-549b8b68d051 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:59.522192 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80e482-32ae-49b7-9b76-549b8b68d051 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:59.527852 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda4436-a33c-4c68-9eca-68d9cbfffcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:59.528550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda4436-a33c-4c68-9eca-68d9cbfffcac admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:59.528828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda4436-a33c-4c68-9eca-68d9cbfffcac admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:59.529130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda4436-a33c-4c68-9eca-68d9cbfffcac admin admin] PATH_INFO: `/v3/projects/4b3cdf29e6cf466ea2bd63de5073aeb5/users/eb2d67d1a43440a29fbd88704945e27b/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:59.547044 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda4436-a33c-4c68-9eca-68d9cbfffcac admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b3cdf29e6cf466ea2bd63de5073aeb5, user_id=eb2d67d1a43440a29fbd88704945e27b, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:59.550107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda4436-a33c-4c68-9eca-68d9cbfffcac admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:59.572514 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 862/1701] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:21:59 2022] PUT /identity/v3/projects/4b3cdf29e6cf466ea2bd63de5073aeb5/users/eb2d67d1a43440a29fbd88704945e27b/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:21:59.584466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351a2f1-8a64-4aea-8b4f-25be5e19bcd6 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:59.584854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351a2f1-8a64-4aea-8b4f-25be5e19bcd6 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:59.585277 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351a2f1-8a64-4aea-8b4f-25be5e19bcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:59.633949 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-8351a2f1-8a64-4aea-8b4f-25be5e19bcd6 None None] MFA Rules not processed for user `eb2d67d1a43440a29fbd88704945e27b`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:21:59.635285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8351a2f1-8a64-4aea-8b4f-25be5e19bcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:59.739189 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 840/1702] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:21:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:21:59.739312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:21:59.751582 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda4436-a33c-4c68-9eca-68d9cbfffcac admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:21:59.753439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8dda4436-a33c-4c68-9eca-68d9cbfffcac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:21:59.835655 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda4436-a33c-4c68-9eca-68d9cbfffcac admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:21:59.840362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1b0c1-d386-442c-8a3f-127a95025e5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:21:59.841065 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1b0c1-d386-442c-8a3f-127a95025e5f service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:21:59.841490 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1b0c1-d386-442c-8a3f-127a95025e5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:21:59.841831 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1b0c1-d386-442c-8a3f-127a95025e5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:21:59.851277 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1b0c1-d386-442c-8a3f-127a95025e5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:21:59.852896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1b0c1-d386-442c-8a3f-127a95025e5f service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:21:59.867708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 863/1703] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:21:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:22:05.493554 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53be65c-5510-4861-a831-a6736c74e33e None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:05.494114 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53be65c-5510-4861-a831-a6736c74e33e None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:05.494114 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53be65c-5510-4861-a831-a6736c74e33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:05.513490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-b53be65c-5510-4861-a831-a6736c74e33e None None] MFA Rules not processed for user `eb2d67d1a43440a29fbd88704945e27b`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:05.514689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-b53be65c-5510-4861-a831-a6736c74e33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:05.522076 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 841/1704] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:09.858562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:15.507886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=5 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:20.078983 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c9651-0f74-4bc8-b674-d71b6d1dca63 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:20.079536 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c9651-0f74-4bc8-b674-d71b6d1dca63 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:20.079536 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c9651-0f74-4bc8-b674-d71b6d1dca63 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:20.087631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 864/1705] 10.222.0.13 () {54 vars in 921 bytes} [Fri Oct 21 21:22:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:22:20.094352 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32f2ff-33d6-4a4a-82ff-8eb0d8ac1802 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:20.094847 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32f2ff-33d6-4a4a-82ff-8eb0d8ac1802 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:20.095140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32f2ff-33d6-4a4a-82ff-8eb0d8ac1802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:20.116781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-5f32f2ff-33d6-4a4a-82ff-8eb0d8ac1802 None None] MFA Rules not processed for user `124f2b5b95e8450ea6867f6868b9e810`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:20.118216 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-5f32f2ff-33d6-4a4a-82ff-8eb0d8ac1802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:20.176391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c9651-0f74-4bc8-b674-d71b6d1dca63 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:20.178507 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ab4c9651-0f74-4bc8-b674-d71b6d1dca63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:20.196024 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 842/1706] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Oct 21 21:22:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:22:20.208805 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32f2ff-33d6-4a4a-82ff-8eb0d8ac1802 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:20.210216 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-5f32f2ff-33d6-4a4a-82ff-8eb0d8ac1802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:20.227483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32f2ff-33d6-4a4a-82ff-8eb0d8ac1802 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:20.233334 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b3303-5053-4b03-966d-3759a3bb45b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:20.233981 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b3303-5053-4b03-966d-3759a3bb45b0 service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:20.235182 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b3303-5053-4b03-966d-3759a3bb45b0 service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:20.235463 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b3303-5053-4b03-966d-3759a3bb45b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:20.245472 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b3303-5053-4b03-966d-3759a3bb45b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:20.247075 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b3303-5053-4b03-966d-3759a3bb45b0 service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:20.258376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 843/1707] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:22:20.268377 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c9651-0f74-4bc8-b674-d71b6d1dca63 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:20.274104 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800d46e-0e57-41bd-bf39-4b5f1012a3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:20.274683 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800d46e-0e57-41bd-bf39-4b5f1012a3dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:20.274962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800d46e-0e57-41bd-bf39-4b5f1012a3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:20.275301 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800d46e-0e57-41bd-bf39-4b5f1012a3dc admin admin] PATH_INFO: `/v3/users/c6e5113e504a4348b74d89a48ba50c17` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:20.276489 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800d46e-0e57-41bd-bf39-4b5f1012a3dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6e5113e504a4348b74d89a48ba50c17)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:20.278431 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800d46e-0e57-41bd-bf39-4b5f1012a3dc admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:20.341479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0800d46e-0e57-41bd-bf39-4b5f1012a3dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6e5113e504a4348b74d89a48ba50c17'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:20.354891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0800d46e-0e57-41bd-bf39-4b5f1012a3dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6e5113e504a4348b74d89a48ba50c17'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:20.363353 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0800d46e-0e57-41bd-bf39-4b5f1012a3dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6e5113e504a4348b74d89a48ba50c17'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:20.364284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0800d46e-0e57-41bd-bf39-4b5f1012a3dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6e5113e504a4348b74d89a48ba50c17'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:20.371803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0800d46e-0e57-41bd-bf39-4b5f1012a3dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6e5113e504a4348b74d89a48ba50c17'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:20.378561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 865/1708] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:22:20 2022] DELETE /identity/v3/users/c6e5113e504a4348b74d89a48ba50c17 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:22:20.565323 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b3303-5053-4b03-966d-3759a3bb45b0 service placement] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:20.579015 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b3303-5053-4b03-966d-3759a3bb45b0 service placement] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:20.646285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6919249-d636-46e9-bd31-5d6b0593adfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:20.647031 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6919249-d636-46e9-bd31-5d6b0593adfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:20.647374 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6919249-d636-46e9-bd31-5d6b0593adfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:20.647744 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6919249-d636-46e9-bd31-5d6b0593adfa admin admin] PATH_INFO: `/v3/projects/b1c0e61355eb4a97b7d19595c466ad3b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:20.649195 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6919249-d636-46e9-bd31-5d6b0593adfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1c0e61355eb4a97b7d19595c466ad3b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:20.651160 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6919249-d636-46e9-bd31-5d6b0593adfa admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:20.699361 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b6919249-d636-46e9-bd31-5d6b0593adfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1c0e61355eb4a97b7d19595c466ad3b', 'request_id': 'req-b6919249-d636-46e9-bd31-5d6b0593adfa'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:20.707973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b6919249-d636-46e9-bd31-5d6b0593adfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1c0e61355eb4a97b7d19595c466ad3b', 'request_id': 'req-b6919249-d636-46e9-bd31-5d6b0593adfa'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:20.713822 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b6919249-d636-46e9-bd31-5d6b0593adfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1c0e61355eb4a97b7d19595c466ad3b', 'request_id': 'req-b6919249-d636-46e9-bd31-5d6b0593adfa'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:20.725058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b6919249-d636-46e9-bd31-5d6b0593adfa admin admin] The token cache is being invalidate because project b1c0e61355eb4a97b7d19595c466ad3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:22:20.725342 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b6919249-d636-46e9-bd31-5d6b0593adfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:20.725976 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b6919249-d636-46e9-bd31-5d6b0593adfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:20.727491 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 844/1709] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:22:20 2022] DELETE /identity/v3/projects/b1c0e61355eb4a97b7d19595c466ad3b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:20.732358 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd05f01-edd4-4982-9be6-905fa67b5b10 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:20.732706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd05f01-edd4-4982-9be6-905fa67b5b10 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:20.733074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd05f01-edd4-4982-9be6-905fa67b5b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:20.753624 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-ecd05f01-edd4-4982-9be6-905fa67b5b10 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:20.754824 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ecd05f01-edd4-4982-9be6-905fa67b5b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:20.812856 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 866/1710] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:20.821166 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150352c-fcba-4a67-b4ea-d7b3352afc3c None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:20.821454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150352c-fcba-4a67-b4ea-d7b3352afc3c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:20.821725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150352c-fcba-4a67-b4ea-d7b3352afc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:20.840814 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-7150352c-fcba-4a67-b4ea-d7b3352afc3c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:20.842011 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-7150352c-fcba-4a67-b4ea-d7b3352afc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:20.850228 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 845/1711] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:20.854284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd05f01-edd4-4982-9be6-905fa67b5b10 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:20.864539 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd05f01-edd4-4982-9be6-905fa67b5b10 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:20.875748 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee977cc8-d89a-4ba6-8ad8-9f3b62c38555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:20.876382 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee977cc8-d89a-4ba6-8ad8-9f3b62c38555 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:20.876663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee977cc8-d89a-4ba6-8ad8-9f3b62c38555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:20.876921 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee977cc8-d89a-4ba6-8ad8-9f3b62c38555 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:20.878194 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee977cc8-d89a-4ba6-8ad8-9f3b62c38555 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:22:20.878528 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee977cc8-d89a-4ba6-8ad8-9f3b62c38555 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:20.880231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee977cc8-d89a-4ba6-8ad8-9f3b62c38555 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:20.892305 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 867/1712] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:22:20.896649 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150352c-fcba-4a67-b4ea-d7b3352afc3c None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:20.906042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150352c-fcba-4a67-b4ea-d7b3352afc3c None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:20.910825 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93100b-2fbb-4ffb-b07e-bf11683e21ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:20.911423 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93100b-2fbb-4ffb-b07e-bf11683e21ab admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:20.911738 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93100b-2fbb-4ffb-b07e-bf11683e21ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:20.912007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93100b-2fbb-4ffb-b07e-bf11683e21ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:20.912628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93100b-2fbb-4ffb-b07e-bf11683e21ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:20.914498 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93100b-2fbb-4ffb-b07e-bf11683e21ab admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:20.918903 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:20.947118 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 846/1713] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:22:20 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:22:20.951243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee977cc8-d89a-4ba6-8ad8-9f3b62c38555 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:20.961790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee977cc8-d89a-4ba6-8ad8-9f3b62c38555 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:20.965304 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:21.021005 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ed2a4-18db-4c4e-b292-a0d5049e704e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:21.021502 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ed2a4-18db-4c4e-b292-a0d5049e704e admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:21.021764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ed2a4-18db-4c4e-b292-a0d5049e704e admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:21.022091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ed2a4-18db-4c4e-b292-a0d5049e704e admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:21.022637 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ed2a4-18db-4c4e-b292-a0d5049e704e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:21.024903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ed2a4-18db-4c4e-b292-a0d5049e704e admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:21.045136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 868/1714] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:22:20 2022] POST /identity/v3/users => generated 302 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:22:21.049353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93100b-2fbb-4ffb-b07e-bf11683e21ab admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:21.059225 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93100b-2fbb-4ffb-b07e-bf11683e21ab admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:21.063716 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2ae0e-aae3-4e13-b479-71d70fe13a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:21.064306 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2ae0e-aae3-4e13-b479-71d70fe13a67 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:21.065102 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2ae0e-aae3-4e13-b479-71d70fe13a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:21.065370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2ae0e-aae3-4e13-b479-71d70fe13a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:21.065996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de2ae0e-aae3-4e13-b479-71d70fe13a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:21.067547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de2ae0e-aae3-4e13-b479-71d70fe13a67 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:21.077412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 847/1715] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:22:21.081606 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ed2a4-18db-4c4e-b292-a0d5049e704e admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:21.092800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ed2a4-18db-4c4e-b292-a0d5049e704e admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:21.097557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ac7cd-1c26-4cba-a8d8-b2a944fc8bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:21.098152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ac7cd-1c26-4cba-a8d8-b2a944fc8bcb admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:21.098428 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ac7cd-1c26-4cba-a8d8-b2a944fc8bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:21.098697 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ac7cd-1c26-4cba-a8d8-b2a944fc8bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:21.099344 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ac7cd-1c26-4cba-a8d8-b2a944fc8bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:21.100981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ac7cd-1c26-4cba-a8d8-b2a944fc8bcb admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:21.111847 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 869/1716] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:22:21.116040 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2ae0e-aae3-4e13-b479-71d70fe13a67 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:21.125860 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2ae0e-aae3-4e13-b479-71d70fe13a67 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:21.130879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc132c2-b353-4fff-a8c2-9a5862bc382c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:21.131519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc132c2-b353-4fff-a8c2-9a5862bc382c admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:21.131826 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc132c2-b353-4fff-a8c2-9a5862bc382c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:21.132099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc132c2-b353-4fff-a8c2-9a5862bc382c admin admin] PATH_INFO: `/v3/projects/ae9147a41fbe4fdea3dd97c636fd7a6e/users/af3e9933f7ac484aaf06536ca673bf3f/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:21.146883 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc132c2-b353-4fff-a8c2-9a5862bc382c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae9147a41fbe4fdea3dd97c636fd7a6e, user_id=af3e9933f7ac484aaf06536ca673bf3f, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:21.150042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc132c2-b353-4fff-a8c2-9a5862bc382c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:21.162489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 848/1717] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:22:21 2022] PUT /identity/v3/projects/ae9147a41fbe4fdea3dd97c636fd7a6e/users/af3e9933f7ac484aaf06536ca673bf3f/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:21.167934 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa1d09-d9ee-4efa-bfa4-baf1a742ece9 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:21.168230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa1d09-d9ee-4efa-bfa4-baf1a742ece9 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:21.168644 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa1d09-d9ee-4efa-bfa4-baf1a742ece9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:21.188762 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-d9aa1d09-d9ee-4efa-bfa4-baf1a742ece9 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:21.190506 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d9aa1d09-d9ee-4efa-bfa4-baf1a742ece9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:21.246992 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 870/1718] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:21.256986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc132c2-b353-4fff-a8c2-9a5862bc382c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:21.258714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-4bc132c2-b353-4fff-a8c2-9a5862bc382c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:21.317644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc132c2-b353-4fff-a8c2-9a5862bc382c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:21.321974 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fd284-ff15-47dd-ac58-4a9885716623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:21.322659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fd284-ff15-47dd-ac58-4a9885716623 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:21.323007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fd284-ff15-47dd-ac58-4a9885716623 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:21.323287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fd284-ff15-47dd-ac58-4a9885716623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:21.334057 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fd284-ff15-47dd-ac58-4a9885716623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:21.335463 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fd284-ff15-47dd-ac58-4a9885716623 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:21.344741 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 849/1719] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:22:22.098787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98145c5-b0e6-491a-8dbd-6dd719db90ba None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:22.099346 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98145c5-b0e6-491a-8dbd-6dd719db90ba None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:22.099346 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98145c5-b0e6-491a-8dbd-6dd719db90ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:22.131072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-f98145c5-b0e6-491a-8dbd-6dd719db90ba None None] MFA Rules not processed for user `af3e9933f7ac484aaf06536ca673bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:22.132328 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f98145c5-b0e6-491a-8dbd-6dd719db90ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:22.198347 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 871/1720] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:22 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:22.203198 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fd284-ff15-47dd-ac58-4a9885716623 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:22.215006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fd284-ff15-47dd-ac58-4a9885716623 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:22.219655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327925c-c47b-4ca5-9f61-b685cfbcbd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:22.220280 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327925c-c47b-4ca5-9f61-b685cfbcbd70 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:22.220543 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327925c-c47b-4ca5-9f61-b685cfbcbd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:22.220850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327925c-c47b-4ca5-9f61-b685cfbcbd70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:22.221485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327925c-c47b-4ca5-9f61-b685cfbcbd70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:22.223375 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327925c-c47b-4ca5-9f61-b685cfbcbd70 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:22.251879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 850/1721] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:22:22 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:22:22.256598 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98145c5-b0e6-491a-8dbd-6dd719db90ba None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:22.267417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98145c5-b0e6-491a-8dbd-6dd719db90ba None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:22.279324 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:22.327628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71ef21-06c0-40cb-bf1c-5c6bcfb2e605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:22.328231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71ef21-06c0-40cb-bf1c-5c6bcfb2e605 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:22.328502 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71ef21-06c0-40cb-bf1c-5c6bcfb2e605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:22.328766 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71ef21-06c0-40cb-bf1c-5c6bcfb2e605 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:22.329378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71ef21-06c0-40cb-bf1c-5c6bcfb2e605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:22.331294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a71ef21-06c0-40cb-bf1c-5c6bcfb2e605 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:22.352212 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 872/1722] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:22:22 2022] POST /identity/v3/users => generated 309 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:22:22.356145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327925c-c47b-4ca5-9f61-b685cfbcbd70 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:22.367947 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327925c-c47b-4ca5-9f61-b685cfbcbd70 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:22.373146 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d9777-ea61-486e-a10c-d52afcb611a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:22.373788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d9777-ea61-486e-a10c-d52afcb611a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:22.374067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d9777-ea61-486e-a10c-d52afcb611a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:22.374344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d9777-ea61-486e-a10c-d52afcb611a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:22.375019 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d9777-ea61-486e-a10c-d52afcb611a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:22.376707 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d9777-ea61-486e-a10c-d52afcb611a3 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:22.388880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 851/1723] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:22:22.393136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71ef21-06c0-40cb-bf1c-5c6bcfb2e605 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:22.403332 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71ef21-06c0-40cb-bf1c-5c6bcfb2e605 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:22.407969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e0599-1795-49ca-8496-41823e827c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:22.408619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e0599-1795-49ca-8496-41823e827c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:22.408899 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e0599-1795-49ca-8496-41823e827c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:22.409167 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e0599-1795-49ca-8496-41823e827c55 admin admin] PATH_INFO: `/v3/projects/b0102e062dd74db589bcba3df6ab220b/users/c4649fc9e2cf499ba9b27fd72e472274/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:22.423525 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e0599-1795-49ca-8496-41823e827c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0102e062dd74db589bcba3df6ab220b, user_id=c4649fc9e2cf499ba9b27fd72e472274, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:22.426730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e0599-1795-49ca-8496-41823e827c55 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:22.439071 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 873/1724] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:22:22 2022] PUT /identity/v3/projects/b0102e062dd74db589bcba3df6ab220b/users/c4649fc9e2cf499ba9b27fd72e472274/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:22.443831 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d9777-ea61-486e-a10c-d52afcb611a3 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:22.454106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d9777-ea61-486e-a10c-d52afcb611a3 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:22.511329 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f2d28-5bae-4e36-8ad5-cda2e10a7176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:22.512194 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f2d28-5bae-4e36-8ad5-cda2e10a7176 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:22.512569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f2d28-5bae-4e36-8ad5-cda2e10a7176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:22.512927 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f2d28-5bae-4e36-8ad5-cda2e10a7176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:22.513710 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f2d28-5bae-4e36-8ad5-cda2e10a7176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:22.515674 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f2d28-5bae-4e36-8ad5-cda2e10a7176 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:22.526806 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 852/1725] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:22:22.531031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e0599-1795-49ca-8496-41823e827c55 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:22.541609 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e0599-1795-49ca-8496-41823e827c55 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:22.546362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be7e68-b08b-4f73-9d51-ba553bc76e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:22.547059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be7e68-b08b-4f73-9d51-ba553bc76e8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:22.547339 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be7e68-b08b-4f73-9d51-ba553bc76e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:22.547655 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be7e68-b08b-4f73-9d51-ba553bc76e8c admin admin] PATH_INFO: `/v3/projects/b0102e062dd74db589bcba3df6ab220b/users/c4649fc9e2cf499ba9b27fd72e472274/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:22.549539 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be7e68-b08b-4f73-9d51-ba553bc76e8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0102e062dd74db589bcba3df6ab220b, user_id=c4649fc9e2cf499ba9b27fd72e472274, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:22.552280 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be7e68-b08b-4f73-9d51-ba553bc76e8c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:22.562919 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 874/1726] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:22:22 2022] PUT /identity/v3/projects/b0102e062dd74db589bcba3df6ab220b/users/c4649fc9e2cf499ba9b27fd72e472274/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:23.225252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9f9e5-4cd4-4933-ae45-c994b517e186 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:23.225781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9f9e5-4cd4-4933-ae45-c994b517e186 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:23.225781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9f9e5-4cd4-4933-ae45-c994b517e186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:23.254180 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-94e9f9e5-4cd4-4933-ae45-c994b517e186 None None] MFA Rules not processed for user `c4649fc9e2cf499ba9b27fd72e472274`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:23.255408 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-94e9f9e5-4cd4-4933-ae45-c994b517e186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:23.259400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:23.337282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 853/1727] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:23 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:24.455081 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bebdb-1d94-4a80-b89c-887344b4ab8e None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:24.456049 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bebdb-1d94-4a80-b89c-887344b4ab8e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:24.456332 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bebdb-1d94-4a80-b89c-887344b4ab8e None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:24.457697 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 875/1728] 10.222.0.13 () {54 vars in 921 bytes} [Fri Oct 21 21:22:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:22:24.488930 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b0527-619a-4b9a-9294-49788cdae9d0 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:24.489400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b0527-619a-4b9a-9294-49788cdae9d0 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:24.489662 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b0527-619a-4b9a-9294-49788cdae9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:24.511249 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-4f0b0527-619a-4b9a-9294-49788cdae9d0 None None] MFA Rules not processed for user `124f2b5b95e8450ea6867f6868b9e810`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:24.513144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-4f0b0527-619a-4b9a-9294-49788cdae9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:24.556520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bebdb-1d94-4a80-b89c-887344b4ab8e None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:24.621428 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 854/1729] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Oct 21 21:22:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:24.633224 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b0527-619a-4b9a-9294-49788cdae9d0 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:24.634593 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-4f0b0527-619a-4b9a-9294-49788cdae9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:24.636222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bebdb-1d94-4a80-b89c-887344b4ab8e None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:24.641500 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b4a8b-f032-40f2-ac21-579dd1786948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:24.642443 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b4a8b-f032-40f2-ac21-579dd1786948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:24.643289 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b4a8b-f032-40f2-ac21-579dd1786948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:24.643583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b4a8b-f032-40f2-ac21-579dd1786948 admin admin] PATH_INFO: `/v3/users/af3e9933f7ac484aaf06536ca673bf3f` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:24.645852 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b4a8b-f032-40f2-ac21-579dd1786948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3e9933f7ac484aaf06536ca673bf3f)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:24.647971 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b0527-619a-4b9a-9294-49788cdae9d0 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:24.648338 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b4a8b-f032-40f2-ac21-579dd1786948 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:24.653586 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdbeca-4b87-48a6-9957-e52865c6bd38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:24.654186 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdbeca-4b87-48a6-9957-e52865c6bd38 service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:24.654465 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdbeca-4b87-48a6-9957-e52865c6bd38 service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:24.654729 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdbeca-4b87-48a6-9957-e52865c6bd38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:24.664006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fdbeca-4b87-48a6-9957-e52865c6bd38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:24.665600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fdbeca-4b87-48a6-9957-e52865c6bd38 service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:24.676504 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 855/1730] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:22:24.710270 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b73b4a8b-f032-40f2-ac21-579dd1786948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3e9933f7ac484aaf06536ca673bf3f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:24.733170 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b73b4a8b-f032-40f2-ac21-579dd1786948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3e9933f7ac484aaf06536ca673bf3f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:24.743061 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:24.748985 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b73b4a8b-f032-40f2-ac21-579dd1786948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3e9933f7ac484aaf06536ca673bf3f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:24.749624 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b73b4a8b-f032-40f2-ac21-579dd1786948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3e9933f7ac484aaf06536ca673bf3f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:24.759681 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b73b4a8b-f032-40f2-ac21-579dd1786948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3e9933f7ac484aaf06536ca673bf3f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:24.767024 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 876/1731] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:22:24 2022] DELETE /identity/v3/users/af3e9933f7ac484aaf06536ca673bf3f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:22:24.943104 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdbeca-4b87-48a6-9957-e52865c6bd38 service placement] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:24.956448 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdbeca-4b87-48a6-9957-e52865c6bd38 service placement] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:25.016164 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79ca1e-d50e-451f-9ad3-d64c481476d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:25.016793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79ca1e-d50e-451f-9ad3-d64c481476d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:25.017065 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79ca1e-d50e-451f-9ad3-d64c481476d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:25.017331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79ca1e-d50e-451f-9ad3-d64c481476d2 admin admin] PATH_INFO: `/v3/projects/ae9147a41fbe4fdea3dd97c636fd7a6e` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:25.018432 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79ca1e-d50e-451f-9ad3-d64c481476d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae9147a41fbe4fdea3dd97c636fd7a6e)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:25.020371 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79ca1e-d50e-451f-9ad3-d64c481476d2 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:25.050963 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:25.068709 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-2f79ca1e-d50e-451f-9ad3-d64c481476d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae9147a41fbe4fdea3dd97c636fd7a6e', 'request_id': 'req-2f79ca1e-d50e-451f-9ad3-d64c481476d2'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:25.076592 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-2f79ca1e-d50e-451f-9ad3-d64c481476d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae9147a41fbe4fdea3dd97c636fd7a6e', 'request_id': 'req-2f79ca1e-d50e-451f-9ad3-d64c481476d2'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:25.083315 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-2f79ca1e-d50e-451f-9ad3-d64c481476d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae9147a41fbe4fdea3dd97c636fd7a6e', 'request_id': 'req-2f79ca1e-d50e-451f-9ad3-d64c481476d2'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:25.095249 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-2f79ca1e-d50e-451f-9ad3-d64c481476d2 admin admin] The token cache is being invalidate because project ae9147a41fbe4fdea3dd97c636fd7a6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:22:25.095529 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-2f79ca1e-d50e-451f-9ad3-d64c481476d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:25.096307 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-2f79ca1e-d50e-451f-9ad3-d64c481476d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:25.097889 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 856/1732] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:22:24 2022] DELETE /identity/v3/projects/ae9147a41fbe4fdea3dd97c636fd7a6e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:25.101964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b4a8b-f032-40f2-ac21-579dd1786948 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:25.103374 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b73b4a8b-f032-40f2-ac21-579dd1786948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:25.117830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b4a8b-f032-40f2-ac21-579dd1786948 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:25.177379 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c698f9-7056-4aa3-8004-2f9802d2c200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:25.178341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c698f9-7056-4aa3-8004-2f9802d2c200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:25.178618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c698f9-7056-4aa3-8004-2f9802d2c200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:25.178891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c698f9-7056-4aa3-8004-2f9802d2c200 admin admin] PATH_INFO: `/v3/users/c4649fc9e2cf499ba9b27fd72e472274` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:25.180068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c698f9-7056-4aa3-8004-2f9802d2c200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4649fc9e2cf499ba9b27fd72e472274)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:25.182043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c698f9-7056-4aa3-8004-2f9802d2c200 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:25.229284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d5c698f9-7056-4aa3-8004-2f9802d2c200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4649fc9e2cf499ba9b27fd72e472274'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:25.242219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d5c698f9-7056-4aa3-8004-2f9802d2c200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4649fc9e2cf499ba9b27fd72e472274'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:25.252261 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d5c698f9-7056-4aa3-8004-2f9802d2c200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4649fc9e2cf499ba9b27fd72e472274'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:25.252925 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d5c698f9-7056-4aa3-8004-2f9802d2c200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4649fc9e2cf499ba9b27fd72e472274'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:25.260523 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d5c698f9-7056-4aa3-8004-2f9802d2c200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4649fc9e2cf499ba9b27fd72e472274'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:25.266447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 877/1733] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:22:25 2022] DELETE /identity/v3/users/c4649fc9e2cf499ba9b27fd72e472274 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:25.440304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79ca1e-d50e-451f-9ad3-d64c481476d2 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:25.441768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2f79ca1e-d50e-451f-9ad3-d64c481476d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:25.455599 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79ca1e-d50e-451f-9ad3-d64c481476d2 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:25.517023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c446d24-637a-4eba-b887-a2c34bdc97b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:25.517553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c446d24-637a-4eba-b887-a2c34bdc97b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:25.518243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c446d24-637a-4eba-b887-a2c34bdc97b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:25.518518 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c446d24-637a-4eba-b887-a2c34bdc97b0 admin admin] PATH_INFO: `/v3/projects/b0102e062dd74db589bcba3df6ab220b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:25.519714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c446d24-637a-4eba-b887-a2c34bdc97b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0102e062dd74db589bcba3df6ab220b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:25.521633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c446d24-637a-4eba-b887-a2c34bdc97b0 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:25.575656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-6c446d24-637a-4eba-b887-a2c34bdc97b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0102e062dd74db589bcba3df6ab220b', 'request_id': 'req-6c446d24-637a-4eba-b887-a2c34bdc97b0'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:25.585500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-6c446d24-637a-4eba-b887-a2c34bdc97b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0102e062dd74db589bcba3df6ab220b', 'request_id': 'req-6c446d24-637a-4eba-b887-a2c34bdc97b0'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:25.593383 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-6c446d24-637a-4eba-b887-a2c34bdc97b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0102e062dd74db589bcba3df6ab220b', 'request_id': 'req-6c446d24-637a-4eba-b887-a2c34bdc97b0'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:25.611694 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-6c446d24-637a-4eba-b887-a2c34bdc97b0 admin admin] The token cache is being invalidate because project b0102e062dd74db589bcba3df6ab220b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:22:25.612079 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-6c446d24-637a-4eba-b887-a2c34bdc97b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:25.612801 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-6c446d24-637a-4eba-b887-a2c34bdc97b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:25.614595 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 857/1734] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:22:25 2022] DELETE /identity/v3/projects/b0102e062dd74db589bcba3df6ab220b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:25.619303 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dfbf3-58aa-48bb-9254-8e865963f2d1 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:25.619832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dfbf3-58aa-48bb-9254-8e865963f2d1 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:25.620041 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dfbf3-58aa-48bb-9254-8e865963f2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:25.641019 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-cc7dfbf3-58aa-48bb-9254-8e865963f2d1 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:25.642238 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-cc7dfbf3-58aa-48bb-9254-8e865963f2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:25.708916 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 878/1735] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:25.716591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715ffdc-bed1-48ed-b7c0-7e73c2c3c296 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:25.716949 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715ffdc-bed1-48ed-b7c0-7e73c2c3c296 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:25.717297 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715ffdc-bed1-48ed-b7c0-7e73c2c3c296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:25.740827 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-2715ffdc-bed1-48ed-b7c0-7e73c2c3c296 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:25.742057 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2715ffdc-bed1-48ed-b7c0-7e73c2c3c296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:25.752822 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 858/1736] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:25.757117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7dfbf3-58aa-48bb-9254-8e865963f2d1 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:25.766863 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7dfbf3-58aa-48bb-9254-8e865963f2d1 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:25.772287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cb7ff-a41f-4644-9c74-b3a285cfc771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:25.772920 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cb7ff-a41f-4644-9c74-b3a285cfc771 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:25.773452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cb7ff-a41f-4644-9c74-b3a285cfc771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:25.773723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cb7ff-a41f-4644-9c74-b3a285cfc771 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:25.774359 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7cb7ff-a41f-4644-9c74-b3a285cfc771 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:22:25.774658 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7cb7ff-a41f-4644-9c74-b3a285cfc771 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:25.776313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7cb7ff-a41f-4644-9c74-b3a285cfc771 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:25.777801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:25.791251 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 879/1737] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:22:25.795011 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715ffdc-bed1-48ed-b7c0-7e73c2c3c296 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:25.804754 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715ffdc-bed1-48ed-b7c0-7e73c2c3c296 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:25.809447 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7791a-4359-462c-879b-a9d51ce6bf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:25.810055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7791a-4359-462c-879b-a9d51ce6bf36 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:25.810334 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7791a-4359-462c-879b-a9d51ce6bf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:25.810609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7791a-4359-462c-879b-a9d51ce6bf36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:25.811257 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7791a-4359-462c-879b-a9d51ce6bf36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:25.813185 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7791a-4359-462c-879b-a9d51ce6bf36 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:25.845658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 859/1738] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:22:25 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:22:25.849802 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cb7ff-a41f-4644-9c74-b3a285cfc771 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:25.859693 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cb7ff-a41f-4644-9c74-b3a285cfc771 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:25.916408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3832633-1ac0-4851-8dbe-5d3625f498db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:25.917042 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3832633-1ac0-4851-8dbe-5d3625f498db admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:25.917336 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3832633-1ac0-4851-8dbe-5d3625f498db admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:25.917611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3832633-1ac0-4851-8dbe-5d3625f498db admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:25.918255 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3832633-1ac0-4851-8dbe-5d3625f498db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:25.920239 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3832633-1ac0-4851-8dbe-5d3625f498db admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:25.943645 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 880/1739] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:22:25 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:22:25.947420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7791a-4359-462c-879b-a9d51ce6bf36 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:25.958571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7791a-4359-462c-879b-a9d51ce6bf36 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:25.964374 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c0c36-b7a5-481a-ba36-20f2680f6340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:25.964933 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c0c36-b7a5-481a-ba36-20f2680f6340 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:25.965213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c0c36-b7a5-481a-ba36-20f2680f6340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:25.965486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c0c36-b7a5-481a-ba36-20f2680f6340 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:25.966128 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c0c36-b7a5-481a-ba36-20f2680f6340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:25.967781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c0c36-b7a5-481a-ba36-20f2680f6340 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:25.978318 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 860/1740] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:22:25.982378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3832633-1ac0-4851-8dbe-5d3625f498db admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:25.993135 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3832633-1ac0-4851-8dbe-5d3625f498db admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:25.998015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fce11c-a1f8-4129-9665-2ea735fa76c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:25.998614 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fce11c-a1f8-4129-9665-2ea735fa76c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:25.998888 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fce11c-a1f8-4129-9665-2ea735fa76c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:25.999156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fce11c-a1f8-4129-9665-2ea735fa76c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:25.999810 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fce11c-a1f8-4129-9665-2ea735fa76c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:26.001401 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fce11c-a1f8-4129-9665-2ea735fa76c4 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:26.012477 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 881/1741] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:22:26.016567 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c0c36-b7a5-481a-ba36-20f2680f6340 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:26.022886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:26.027464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c0c36-b7a5-481a-ba36-20f2680f6340 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:26.035058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4b3bd-bf50-408e-b917-f67522b2f760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:26.035770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4b3bd-bf50-408e-b917-f67522b2f760 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:26.036055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4b3bd-bf50-408e-b917-f67522b2f760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:26.036324 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4b3bd-bf50-408e-b917-f67522b2f760 admin admin] PATH_INFO: `/v3/projects/7794795acaa34930acc4c19ac68a05fe/users/ba86ca4262134d7b876e0d0903692188/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:26.051692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4b3bd-bf50-408e-b917-f67522b2f760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7794795acaa34930acc4c19ac68a05fe, user_id=ba86ca4262134d7b876e0d0903692188, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:26.054317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4b3bd-bf50-408e-b917-f67522b2f760 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:26.067719 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 861/1742] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:22:26 2022] PUT /identity/v3/projects/7794795acaa34930acc4c19ac68a05fe/users/ba86ca4262134d7b876e0d0903692188/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:26.078905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a25c8-b84a-49be-be06-079993790767 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:26.079191 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a25c8-b84a-49be-be06-079993790767 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:26.079469 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a25c8-b84a-49be-be06-079993790767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:26.107352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-eb5a25c8-b84a-49be-be06-079993790767 None None] MFA Rules not processed for user `ba86ca4262134d7b876e0d0903692188`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:26.108764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-eb5a25c8-b84a-49be-be06-079993790767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:26.173763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 882/1743] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:26.178178 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4b3bd-bf50-408e-b917-f67522b2f760 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:26.240128 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4b3bd-bf50-408e-b917-f67522b2f760 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:26.245073 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42094d-1029-4724-85d0-b23890660564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:26.245696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42094d-1029-4724-85d0-b23890660564 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:26.245978 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42094d-1029-4724-85d0-b23890660564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:26.246250 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42094d-1029-4724-85d0-b23890660564 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:26.247024 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e42094d-1029-4724-85d0-b23890660564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:26.249005 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e42094d-1029-4724-85d0-b23890660564 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:26.285386 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 862/1744] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:22:26 2022] POST /identity/v3/projects => generated 391 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:22:26.286598 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a25c8-b84a-49be-be06-079993790767 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:26.299163 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a25c8-b84a-49be-be06-079993790767 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:26.378040 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc4037-ddac-4787-8ede-a9b2fee67375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:26.378682 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc4037-ddac-4787-8ede-a9b2fee67375 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:26.378953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc4037-ddac-4787-8ede-a9b2fee67375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:26.379220 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc4037-ddac-4787-8ede-a9b2fee67375 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:26.379877 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc4037-ddac-4787-8ede-a9b2fee67375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:26.381801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc4037-ddac-4787-8ede-a9b2fee67375 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:26.392295 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:26.415065 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 883/1745] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:22:26 2022] POST /identity/v3/users => generated 302 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:22:26.419455 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42094d-1029-4724-85d0-b23890660564 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:26.431831 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42094d-1029-4724-85d0-b23890660564 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:26.436745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046419d9-3d90-49b6-8aa8-3479c6375fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:26.437357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046419d9-3d90-49b6-8aa8-3479c6375fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:26.437635 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046419d9-3d90-49b6-8aa8-3479c6375fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:26.437904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046419d9-3d90-49b6-8aa8-3479c6375fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:26.439049 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046419d9-3d90-49b6-8aa8-3479c6375fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:26.441337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046419d9-3d90-49b6-8aa8-3479c6375fc7 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:26.456137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 863/1746] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:22:26.457231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc4037-ddac-4787-8ede-a9b2fee67375 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:26.467477 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc4037-ddac-4787-8ede-a9b2fee67375 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:26.473174 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd058c-b7be-4083-84b4-2addb8ad4426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:26.473860 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd058c-b7be-4083-84b4-2addb8ad4426 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:26.474207 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd058c-b7be-4083-84b4-2addb8ad4426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:26.474567 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd058c-b7be-4083-84b4-2addb8ad4426 admin admin] PATH_INFO: `/v3/projects/803062c1482847f6b7539ab450cb9a97/users/e6fe472754e642629344f443e7c0acc7/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:26.490556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd058c-b7be-4083-84b4-2addb8ad4426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=803062c1482847f6b7539ab450cb9a97, user_id=e6fe472754e642629344f443e7c0acc7, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:26.493705 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd058c-b7be-4083-84b4-2addb8ad4426 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:26.510943 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 884/1747] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:22:26 2022] PUT /identity/v3/projects/803062c1482847f6b7539ab450cb9a97/users/e6fe472754e642629344f443e7c0acc7/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:26.515010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046419d9-3d90-49b6-8aa8-3479c6375fc7 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:26.527119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046419d9-3d90-49b6-8aa8-3479c6375fc7 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:26.584190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e11f6d-e0d2-49eb-a722-9340de0f1cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:26.584849 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e11f6d-e0d2-49eb-a722-9340de0f1cac admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:26.585124 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e11f6d-e0d2-49eb-a722-9340de0f1cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:26.585390 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e11f6d-e0d2-49eb-a722-9340de0f1cac admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:26.586045 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e11f6d-e0d2-49eb-a722-9340de0f1cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:26.587791 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e11f6d-e0d2-49eb-a722-9340de0f1cac admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:26.598167 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 864/1748] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:22:26.602388 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd058c-b7be-4083-84b4-2addb8ad4426 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:26.613558 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd058c-b7be-4083-84b4-2addb8ad4426 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:26.618192 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fa74e-c63c-4087-a34d-4f7ea6e86c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:26.618849 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fa74e-c63c-4087-a34d-4f7ea6e86c47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:26.619132 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fa74e-c63c-4087-a34d-4f7ea6e86c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:26.619423 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fa74e-c63c-4087-a34d-4f7ea6e86c47 admin admin] PATH_INFO: `/v3/projects/803062c1482847f6b7539ab450cb9a97/users/e6fe472754e642629344f443e7c0acc7/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:26.621330 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140fa74e-c63c-4087-a34d-4f7ea6e86c47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=803062c1482847f6b7539ab450cb9a97, user_id=e6fe472754e642629344f443e7c0acc7, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:26.624017 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140fa74e-c63c-4087-a34d-4f7ea6e86c47 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:26.634091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 885/1749] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:22:26 2022] PUT /identity/v3/projects/803062c1482847f6b7539ab450cb9a97/users/e6fe472754e642629344f443e7c0acc7/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:26.644622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb70481-7808-4da4-a42e-7b94e0620fc8 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:26.644913 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb70481-7808-4da4-a42e-7b94e0620fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:26.645213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb70481-7808-4da4-a42e-7b94e0620fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:26.673661 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-3fb70481-7808-4da4-a42e-7b94e0620fc8 None None] MFA Rules not processed for user `e6fe472754e642629344f443e7c0acc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:26.674934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3fb70481-7808-4da4-a42e-7b94e0620fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:26.715792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:26.744468 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 865/1750] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:26 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:26.755183 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fa74e-c63c-4087-a34d-4f7ea6e86c47 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:26.756903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-140fa74e-c63c-4087-a34d-4f7ea6e86c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:26.816448 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fa74e-c63c-4087-a34d-4f7ea6e86c47 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:26.820706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df13bbf-a6b1-4b33-94b4-ab44eac877a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:26.821396 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df13bbf-a6b1-4b33-94b4-ab44eac877a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:26.821763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df13bbf-a6b1-4b33-94b4-ab44eac877a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:26.822139 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df13bbf-a6b1-4b33-94b4-ab44eac877a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:26.833190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df13bbf-a6b1-4b33-94b4-ab44eac877a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:26.834849 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df13bbf-a6b1-4b33-94b4-ab44eac877a8 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:26.845184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 886/1751] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:26 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:22:26.904219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb70481-7808-4da4-a42e-7b94e0620fc8 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:26.905666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3fb70481-7808-4da4-a42e-7b94e0620fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:26.917799 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb70481-7808-4da4-a42e-7b94e0620fc8 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:26.921440 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4358941-86d6-4538-86b1-5f982728b7a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:26.922053 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4358941-86d6-4538-86b1-5f982728b7a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:26.922330 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4358941-86d6-4538-86b1-5f982728b7a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:26.922603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4358941-86d6-4538-86b1-5f982728b7a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:26.978342 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4358941-86d6-4538-86b1-5f982728b7a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:26.979793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4358941-86d6-4538-86b1-5f982728b7a9 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:26.989114 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 866/1752] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:26 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:22:32.437427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df13bbf-a6b1-4b33-94b4-ab44eac877a8 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:32.497454 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df13bbf-a6b1-4b33-94b4-ab44eac877a8 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:32.501888 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc8b7c-78d0-44b1-85bb-1209f6da885b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:32.502494 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc8b7c-78d0-44b1-85bb-1209f6da885b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:32.502768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc8b7c-78d0-44b1-85bb-1209f6da885b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:32.503035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc8b7c-78d0-44b1-85bb-1209f6da885b admin admin] PATH_INFO: `/v3/users/ba86ca4262134d7b876e0d0903692188` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:32.504145 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc8b7c-78d0-44b1-85bb-1209f6da885b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba86ca4262134d7b876e0d0903692188)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:32.506045 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc8b7c-78d0-44b1-85bb-1209f6da885b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:32.550940 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-36dc8b7c-78d0-44b1-85bb-1209f6da885b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba86ca4262134d7b876e0d0903692188'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:32.558448 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:32.563760 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-36dc8b7c-78d0-44b1-85bb-1209f6da885b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba86ca4262134d7b876e0d0903692188'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:32.573183 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-36dc8b7c-78d0-44b1-85bb-1209f6da885b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba86ca4262134d7b876e0d0903692188'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:32.573812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-36dc8b7c-78d0-44b1-85bb-1209f6da885b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba86ca4262134d7b876e0d0903692188'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:32.581182 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-36dc8b7c-78d0-44b1-85bb-1209f6da885b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba86ca4262134d7b876e0d0903692188'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:32.586897 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 887/1753] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:22:32 2022] DELETE /identity/v3/users/ba86ca4262134d7b876e0d0903692188 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:32.591560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cfd74-b9a0-49a1-b0a0-e6178a996da9 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:32.592051 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cfd74-b9a0-49a1-b0a0-e6178a996da9 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:32.592145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cfd74-b9a0-49a1-b0a0-e6178a996da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:32.610548 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-f73cfd74-b9a0-49a1-b0a0-e6178a996da9 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:32.612256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-f73cfd74-b9a0-49a1-b0a0-e6178a996da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:32.676150 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 867/1754] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:32.685973 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc8b7c-78d0-44b1-85bb-1209f6da885b admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:32.687351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-36dc8b7c-78d0-44b1-85bb-1209f6da885b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:32.751182 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc8b7c-78d0-44b1-85bb-1209f6da885b admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:32.755365 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b6c36-6bcf-47eb-a7d4-388414769639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:32.756018 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b6c36-6bcf-47eb-a7d4-388414769639 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:32.756298 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b6c36-6bcf-47eb-a7d4-388414769639 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:32.756564 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b6c36-6bcf-47eb-a7d4-388414769639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:32.765658 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b6c36-6bcf-47eb-a7d4-388414769639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:32.767091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b6c36-6bcf-47eb-a7d4-388414769639 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:32.777945 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 888/1755] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:22:33.185595 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cfd74-b9a0-49a1-b0a0-e6178a996da9 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:33.187185 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-f73cfd74-b9a0-49a1-b0a0-e6178a996da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:33.200181 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cfd74-b9a0-49a1-b0a0-e6178a996da9 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:33.204813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3353015-f103-427d-832f-014e7fb258e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:33.206092 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3353015-f103-427d-832f-014e7fb258e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:33.206486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3353015-f103-427d-832f-014e7fb258e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:33.206759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3353015-f103-427d-832f-014e7fb258e7 admin admin] PATH_INFO: `/v3/projects/7794795acaa34930acc4c19ac68a05fe` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:33.207976 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3353015-f103-427d-832f-014e7fb258e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7794795acaa34930acc4c19ac68a05fe)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:33.210222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3353015-f103-427d-832f-014e7fb258e7 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:33.260651 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e3353015-f103-427d-832f-014e7fb258e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7794795acaa34930acc4c19ac68a05fe', 'request_id': 'req-e3353015-f103-427d-832f-014e7fb258e7'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:33.269414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e3353015-f103-427d-832f-014e7fb258e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7794795acaa34930acc4c19ac68a05fe', 'request_id': 'req-e3353015-f103-427d-832f-014e7fb258e7'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:33.276264 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e3353015-f103-427d-832f-014e7fb258e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7794795acaa34930acc4c19ac68a05fe', 'request_id': 'req-e3353015-f103-427d-832f-014e7fb258e7'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:33.284247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:33.289389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e3353015-f103-427d-832f-014e7fb258e7 admin admin] The token cache is being invalidate because project 7794795acaa34930acc4c19ac68a05fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:22:33.289836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e3353015-f103-427d-832f-014e7fb258e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:33.291213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e3353015-f103-427d-832f-014e7fb258e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:33.294437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 868/1756] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:22:33 2022] DELETE /identity/v3/projects/7794795acaa34930acc4c19ac68a05fe => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:33.300819 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b6c36-6bcf-47eb-a7d4-388414769639 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:33.310610 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b6c36-6bcf-47eb-a7d4-388414769639 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:33.369765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77770196-e812-4548-9824-d1c4534df9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:33.370396 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77770196-e812-4548-9824-d1c4534df9fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:33.370671 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77770196-e812-4548-9824-d1c4534df9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:33.370938 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77770196-e812-4548-9824-d1c4534df9fb admin admin] PATH_INFO: `/v3/users/e6fe472754e642629344f443e7c0acc7` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:33.372054 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77770196-e812-4548-9824-d1c4534df9fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6fe472754e642629344f443e7c0acc7)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:33.373998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77770196-e812-4548-9824-d1c4534df9fb admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:33.427360 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-77770196-e812-4548-9824-d1c4534df9fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6fe472754e642629344f443e7c0acc7'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:33.439705 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-77770196-e812-4548-9824-d1c4534df9fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6fe472754e642629344f443e7c0acc7'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:33.447320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-77770196-e812-4548-9824-d1c4534df9fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6fe472754e642629344f443e7c0acc7'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:33.448141 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-77770196-e812-4548-9824-d1c4534df9fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6fe472754e642629344f443e7c0acc7'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:33.455578 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-77770196-e812-4548-9824-d1c4534df9fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6fe472754e642629344f443e7c0acc7'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:33.456989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:33.463440 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 889/1757] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:22:33 2022] DELETE /identity/v3/users/e6fe472754e642629344f443e7c0acc7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:22:33.743430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3353015-f103-427d-832f-014e7fb258e7 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:33.744973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-e3353015-f103-427d-832f-014e7fb258e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:33.758456 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3353015-f103-427d-832f-014e7fb258e7 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:33.820582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d6a57-492a-4abb-9c17-d3afbad0c84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:33.821278 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d6a57-492a-4abb-9c17-d3afbad0c84e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:33.821791 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d6a57-492a-4abb-9c17-d3afbad0c84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:33.821961 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d6a57-492a-4abb-9c17-d3afbad0c84e admin admin] PATH_INFO: `/v3/projects/803062c1482847f6b7539ab450cb9a97` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:33.823184 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d6a57-492a-4abb-9c17-d3afbad0c84e admin admin] RBAC: Authorizing `identity:delete_project(project_id=803062c1482847f6b7539ab450cb9a97)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:33.825296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d6a57-492a-4abb-9c17-d3afbad0c84e admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:33.900547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-6d2d6a57-492a-4abb-9c17-d3afbad0c84e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '803062c1482847f6b7539ab450cb9a97', 'request_id': 'req-6d2d6a57-492a-4abb-9c17-d3afbad0c84e'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:33.908861 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-6d2d6a57-492a-4abb-9c17-d3afbad0c84e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '803062c1482847f6b7539ab450cb9a97', 'request_id': 'req-6d2d6a57-492a-4abb-9c17-d3afbad0c84e'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:33.915769 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-6d2d6a57-492a-4abb-9c17-d3afbad0c84e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '803062c1482847f6b7539ab450cb9a97', 'request_id': 'req-6d2d6a57-492a-4abb-9c17-d3afbad0c84e'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:33.928114 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-6d2d6a57-492a-4abb-9c17-d3afbad0c84e admin admin] The token cache is being invalidate because project 803062c1482847f6b7539ab450cb9a97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:22:33.928400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-6d2d6a57-492a-4abb-9c17-d3afbad0c84e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:33.929229 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-6d2d6a57-492a-4abb-9c17-d3afbad0c84e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:33.931262 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 869/1758] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:22:33 2022] DELETE /identity/v3/projects/803062c1482847f6b7539ab450cb9a97 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:33.936385 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e56ed-076c-4db2-a099-97f8e58d966e None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:33.936672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e56ed-076c-4db2-a099-97f8e58d966e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:33.936944 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e56ed-076c-4db2-a099-97f8e58d966e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:33.960209 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-2c5e56ed-076c-4db2-a099-97f8e58d966e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:33.961342 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2c5e56ed-076c-4db2-a099-97f8e58d966e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:34.035941 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 890/1759] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:34.042949 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5420e12-10e6-496a-bdae-399e3360e885 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:34.043252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5420e12-10e6-496a-bdae-399e3360e885 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:34.043502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5420e12-10e6-496a-bdae-399e3360e885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:34.066217 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-c5420e12-10e6-496a-bdae-399e3360e885 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:34.067948 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c5420e12-10e6-496a-bdae-399e3360e885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:34.078352 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 870/1760] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:34.082107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e56ed-076c-4db2-a099-97f8e58d966e None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:34.099240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e56ed-076c-4db2-a099-97f8e58d966e None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:34.104932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521140dc-d228-4687-b2bc-b36d42fbbf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:34.105572 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521140dc-d228-4687-b2bc-b36d42fbbf73 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:34.105852 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521140dc-d228-4687-b2bc-b36d42fbbf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:34.106285 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521140dc-d228-4687-b2bc-b36d42fbbf73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:34.106951 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521140dc-d228-4687-b2bc-b36d42fbbf73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:22:34.107252 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521140dc-d228-4687-b2bc-b36d42fbbf73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:34.110799 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521140dc-d228-4687-b2bc-b36d42fbbf73 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:34.138855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 891/1761] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:22:34.145021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5420e12-10e6-496a-bdae-399e3360e885 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:34.157109 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5420e12-10e6-496a-bdae-399e3360e885 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:34.162358 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d3a1d-5582-4b7b-a697-b6ad2999baee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:34.162994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d3a1d-5582-4b7b-a697-b6ad2999baee admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:34.163274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d3a1d-5582-4b7b-a697-b6ad2999baee admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:34.163549 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d3a1d-5582-4b7b-a697-b6ad2999baee admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:34.164228 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d3a1d-5582-4b7b-a697-b6ad2999baee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:34.166178 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d3a1d-5582-4b7b-a697-b6ad2999baee admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:34.197079 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 871/1762] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:22:34 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:22:34.201838 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521140dc-d228-4687-b2bc-b36d42fbbf73 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:34.261769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521140dc-d228-4687-b2bc-b36d42fbbf73 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:34.372191 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36de3c-a6b8-43ab-8ff6-1bd922388e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:34.373106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36de3c-a6b8-43ab-8ff6-1bd922388e07 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:34.373392 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36de3c-a6b8-43ab-8ff6-1bd922388e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:34.373662 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36de3c-a6b8-43ab-8ff6-1bd922388e07 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:34.374540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36de3c-a6b8-43ab-8ff6-1bd922388e07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:34.376640 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36de3c-a6b8-43ab-8ff6-1bd922388e07 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:34.405187 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 892/1763] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:22:34 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:22:34.409209 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d3a1d-5582-4b7b-a697-b6ad2999baee admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:34.420587 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d3a1d-5582-4b7b-a697-b6ad2999baee admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:34.427331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2d90d-afff-40c7-a12a-be84f9cdd4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:34.428069 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2d90d-afff-40c7-a12a-be84f9cdd4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:34.428485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2d90d-afff-40c7-a12a-be84f9cdd4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:34.428833 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2d90d-afff-40c7-a12a-be84f9cdd4a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:34.430048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2d90d-afff-40c7-a12a-be84f9cdd4a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:34.431778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2d90d-afff-40c7-a12a-be84f9cdd4a2 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:34.443731 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 872/1764] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:22:34.447503 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36de3c-a6b8-43ab-8ff6-1bd922388e07 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:34.469578 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36de3c-a6b8-43ab-8ff6-1bd922388e07 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:34.474853 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7fe8e-bcd0-4746-ab28-f31270c1e9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:34.475842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7fe8e-bcd0-4746-ab28-f31270c1e9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:34.476274 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7fe8e-bcd0-4746-ab28-f31270c1e9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:34.476731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7fe8e-bcd0-4746-ab28-f31270c1e9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:34.477428 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7fe8e-bcd0-4746-ab28-f31270c1e9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:34.479427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7fe8e-bcd0-4746-ab28-f31270c1e9f9 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:34.494059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 893/1765] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:22:34.495065 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2d90d-afff-40c7-a12a-be84f9cdd4a2 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:34.518234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2d90d-afff-40c7-a12a-be84f9cdd4a2 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:34.534840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ec9c5-3048-433d-9d6b-52b864a80c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:34.535459 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ec9c5-3048-433d-9d6b-52b864a80c7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:34.535783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ec9c5-3048-433d-9d6b-52b864a80c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:34.536060 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ec9c5-3048-433d-9d6b-52b864a80c7b admin admin] PATH_INFO: `/v3/projects/cb9e2b3b4ea14dd7b742ed23fdf2c43c/users/d3e8b787756244bd85449ec4e6018939/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:34.551518 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ec9c5-3048-433d-9d6b-52b864a80c7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb9e2b3b4ea14dd7b742ed23fdf2c43c, user_id=d3e8b787756244bd85449ec4e6018939, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:34.554186 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ec9c5-3048-433d-9d6b-52b864a80c7b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:34.568095 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 873/1766] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:22:34 2022] PUT /identity/v3/projects/cb9e2b3b4ea14dd7b742ed23fdf2c43c/users/d3e8b787756244bd85449ec4e6018939/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:22:34.579452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9199d2d-833c-4d85-b7de-de94cd3bce67 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:34.579767 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9199d2d-833c-4d85-b7de-de94cd3bce67 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:34.580035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9199d2d-833c-4d85-b7de-de94cd3bce67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:34.608160 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-f9199d2d-833c-4d85-b7de-de94cd3bce67 None None] MFA Rules not processed for user `d3e8b787756244bd85449ec4e6018939`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:34.609456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f9199d2d-833c-4d85-b7de-de94cd3bce67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:34.666310 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:34.677602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 894/1767] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:34.681757 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ec9c5-3048-433d-9d6b-52b864a80c7b admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:34.710165 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:34.752240 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ec9c5-3048-433d-9d6b-52b864a80c7b admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:34.757119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44146d-42e8-40fa-860f-f86a58afc381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:34.757729 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44146d-42e8-40fa-860f-f86a58afc381 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:34.758011 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44146d-42e8-40fa-860f-f86a58afc381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:34.758283 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44146d-42e8-40fa-860f-f86a58afc381 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:34.758917 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44146d-42e8-40fa-860f-f86a58afc381 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:34.760913 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44146d-42e8-40fa-860f-f86a58afc381 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:34.790517 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 874/1768] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:22:34 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:22:34.794583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9199d2d-833c-4d85-b7de-de94cd3bce67 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:34.806459 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9199d2d-833c-4d85-b7de-de94cd3bce67 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:34.863393 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a638373-08e4-4d38-9573-e1d54cfc9197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:34.864032 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a638373-08e4-4d38-9573-e1d54cfc9197 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:34.864328 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a638373-08e4-4d38-9573-e1d54cfc9197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:34.864580 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a638373-08e4-4d38-9573-e1d54cfc9197 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:34.865210 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a638373-08e4-4d38-9573-e1d54cfc9197 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:34.867156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a638373-08e4-4d38-9573-e1d54cfc9197 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:34.888034 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 895/1769] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:22:34 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:22:34.891967 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44146d-42e8-40fa-860f-f86a58afc381 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:34.902361 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44146d-42e8-40fa-860f-f86a58afc381 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:34.906930 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d31d3-6806-4076-ab34-f40825218e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:34.907523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d31d3-6806-4076-ab34-f40825218e45 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:34.907836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d31d3-6806-4076-ab34-f40825218e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:34.908134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d31d3-6806-4076-ab34-f40825218e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:34.908769 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063d31d3-6806-4076-ab34-f40825218e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:34.910349 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063d31d3-6806-4076-ab34-f40825218e45 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:34.920597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 875/1770] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:22:34.925203 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a638373-08e4-4d38-9573-e1d54cfc9197 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:34.935176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a638373-08e4-4d38-9573-e1d54cfc9197 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:34.940274 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3858a-c64b-41d2-b863-49827b409f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:34.940907 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3858a-c64b-41d2-b863-49827b409f51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:34.941182 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3858a-c64b-41d2-b863-49827b409f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:34.941448 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3858a-c64b-41d2-b863-49827b409f51 admin admin] PATH_INFO: `/v3/projects/23889aa3ceee46b3acf80a6610c36c82/users/87a7096021494a10b681f8c6b4ac734f/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:34.955750 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3858a-c64b-41d2-b863-49827b409f51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23889aa3ceee46b3acf80a6610c36c82, user_id=87a7096021494a10b681f8c6b4ac734f, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:34.958384 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3858a-c64b-41d2-b863-49827b409f51 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:34.970948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 896/1771] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:22:34 2022] PUT /identity/v3/projects/23889aa3ceee46b3acf80a6610c36c82/users/87a7096021494a10b681f8c6b4ac734f/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:34.975220 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d31d3-6806-4076-ab34-f40825218e45 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:34.985016 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d31d3-6806-4076-ab34-f40825218e45 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:35.042991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e249e-bd16-41a9-b559-f8c8ed6db2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:35.043625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e249e-bd16-41a9-b559-f8c8ed6db2a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:35.043925 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e249e-bd16-41a9-b559-f8c8ed6db2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:35.044196 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e249e-bd16-41a9-b559-f8c8ed6db2a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:35.044975 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e249e-bd16-41a9-b559-f8c8ed6db2a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:35.046580 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e249e-bd16-41a9-b559-f8c8ed6db2a8 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:35.056828 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 876/1772] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:22:35.060956 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3858a-c64b-41d2-b863-49827b409f51 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:35.071595 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3858a-c64b-41d2-b863-49827b409f51 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:35.076576 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e8884-48cd-4d22-af1b-9ce77fd8bd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:35.077306 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e8884-48cd-4d22-af1b-9ce77fd8bd1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:35.077651 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e8884-48cd-4d22-af1b-9ce77fd8bd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:35.077990 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e8884-48cd-4d22-af1b-9ce77fd8bd1d admin admin] PATH_INFO: `/v3/projects/23889aa3ceee46b3acf80a6610c36c82/users/87a7096021494a10b681f8c6b4ac734f/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:35.080079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e8884-48cd-4d22-af1b-9ce77fd8bd1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=23889aa3ceee46b3acf80a6610c36c82, user_id=87a7096021494a10b681f8c6b4ac734f, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:35.082829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e8884-48cd-4d22-af1b-9ce77fd8bd1d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:35.094775 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 897/1773] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:22:35 2022] PUT /identity/v3/projects/23889aa3ceee46b3acf80a6610c36c82/users/87a7096021494a10b681f8c6b4ac734f/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:35.104771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69b260-b2e3-4f39-bbad-cf508cede959 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:35.105054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69b260-b2e3-4f39-bbad-cf508cede959 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:35.105324 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69b260-b2e3-4f39-bbad-cf508cede959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:35.134393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-5e69b260-b2e3-4f39-bbad-cf508cede959 None None] MFA Rules not processed for user `87a7096021494a10b681f8c6b4ac734f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:35.135727 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-5e69b260-b2e3-4f39-bbad-cf508cede959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:35.205796 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 877/1774] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:35.216743 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e8884-48cd-4d22-af1b-9ce77fd8bd1d admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:35.219034 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ab8e8884-48cd-4d22-af1b-9ce77fd8bd1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:35.281170 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e8884-48cd-4d22-af1b-9ce77fd8bd1d admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:35.285117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ae00e-4282-4453-ada6-5f6b60685390 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:35.285745 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ae00e-4282-4453-ada6-5f6b60685390 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:35.286085 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ae00e-4282-4453-ada6-5f6b60685390 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:35.286370 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ae00e-4282-4453-ada6-5f6b60685390 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:35.345272 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ae00e-4282-4453-ada6-5f6b60685390 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:35.346692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ae00e-4282-4453-ada6-5f6b60685390 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:35.351191 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:35.357124 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 898/1775] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:22:35.574782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ee047-3248-4f98-ab66-be412abc60a3 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:35.575067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ee047-3248-4f98-ab66-be412abc60a3 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:35.575364 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ee047-3248-4f98-ab66-be412abc60a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:35.597686 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-233ee047-3248-4f98-ab66-be412abc60a3 None None] MFA Rules not processed for user `2346003afe74477496045e9bc72173d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:35.598907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-233ee047-3248-4f98-ab66-be412abc60a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:35.613512 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:35.677916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 878/1776] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Oct 21 21:22:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:22:35.734469 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ae00e-4282-4453-ada6-5f6b60685390 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:35.736656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8b5ae00e-4282-4453-ada6-5f6b60685390 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:35.841811 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ae00e-4282-4453-ada6-5f6b60685390 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:35.847131 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd17f2-ec67-46e5-abeb-1084cdac62de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ce720c8df014f9a8af062138aecd1b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:35.847849 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd17f2-ec67-46e5-abeb-1084cdac62de service swift] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:35.848194 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd17f2-ec67-46e5-abeb-1084cdac62de service swift] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:35.848555 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd17f2-ec67-46e5-abeb-1084cdac62de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:35.856839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd17f2-ec67-46e5-abeb-1084cdac62de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:35.858498 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd17f2-ec67-46e5-abeb-1084cdac62de service swift] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:35.867415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 899/1777] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Oct 21 21:22:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 21:22:36.384624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac2232-6a6f-493f-90c9-268489d05858 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:36.385079 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac2232-6a6f-493f-90c9-268489d05858 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:36.385303 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac2232-6a6f-493f-90c9-268489d05858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:36.405061 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-c4ac2232-6a6f-493f-90c9-268489d05858 None None] MFA Rules not processed for user `2346003afe74477496045e9bc72173d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:36.406294 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c4ac2232-6a6f-493f-90c9-268489d05858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:36.415224 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 879/1778] 10.222.0.13 () {58 vars in 967 bytes} [Fri Oct 21 21:22:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:36.429369 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd17f2-ec67-46e5-abeb-1084cdac62de service swift] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:36.444851 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd17f2-ec67-46e5-abeb-1084cdac62de service swift] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:36.450365 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694661ea-fd8d-4ef9-97d7-9efd39695c27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ce720c8df014f9a8af062138aecd1b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:36.451040 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694661ea-fd8d-4ef9-97d7-9efd39695c27 service swift] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:36.451392 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694661ea-fd8d-4ef9-97d7-9efd39695c27 service swift] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:36.451777 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694661ea-fd8d-4ef9-97d7-9efd39695c27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:36.460322 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694661ea-fd8d-4ef9-97d7-9efd39695c27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:36.461971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694661ea-fd8d-4ef9-97d7-9efd39695c27 service swift] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:36.476704 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 900/1779] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Oct 21 21:22:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 21:22:36.719548 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b959f-6fda-479a-a614-d4a9b38d78da None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:36.719863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b959f-6fda-479a-a614-d4a9b38d78da None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:36.720134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b959f-6fda-479a-a614-d4a9b38d78da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:36.743263 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-335b959f-6fda-479a-a614-d4a9b38d78da None None] MFA Rules not processed for user `2346003afe74477496045e9bc72173d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:36.744513 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-335b959f-6fda-479a-a614-d4a9b38d78da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:36.752328 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 880/1780] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Oct 21 21:22:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:36.764392 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694661ea-fd8d-4ef9-97d7-9efd39695c27 service swift] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:36.776156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694661ea-fd8d-4ef9-97d7-9efd39695c27 service swift] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:36.781710 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bb71d-3ba0-4fb0-ae4a-0d7282b38913 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ce720c8df014f9a8af062138aecd1b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:36.782529 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bb71d-3ba0-4fb0-ae4a-0d7282b38913 service swift] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:36.782812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bb71d-3ba0-4fb0-ae4a-0d7282b38913 service swift] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:36.783120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bb71d-3ba0-4fb0-ae4a-0d7282b38913 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:36.791533 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bb71d-3ba0-4fb0-ae4a-0d7282b38913 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:36.793328 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bb71d-3ba0-4fb0-ae4a-0d7282b38913 service swift] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:36.801417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 901/1781] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Oct 21 21:22:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 21:22:37.155879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027335bb-ca96-4117-ad4a-d277abc0e90a None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:37.156153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027335bb-ca96-4117-ad4a-d277abc0e90a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:37.156424 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027335bb-ca96-4117-ad4a-d277abc0e90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:37.175782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-027335bb-ca96-4117-ad4a-d277abc0e90a None None] MFA Rules not processed for user `2346003afe74477496045e9bc72173d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:37.176906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-027335bb-ca96-4117-ad4a-d277abc0e90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:37.183913 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 881/1782] 10.222.0.13 () {58 vars in 967 bytes} [Fri Oct 21 21:22:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:37.196377 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bb71d-3ba0-4fb0-ae4a-0d7282b38913 service swift] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:37.206518 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bb71d-3ba0-4fb0-ae4a-0d7282b38913 service swift] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:37.211400 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc67332-8a60-4d13-81e8-898ff2165750 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ce720c8df014f9a8af062138aecd1b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:37.212026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc67332-8a60-4d13-81e8-898ff2165750 service swift] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:37.212304 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc67332-8a60-4d13-81e8-898ff2165750 service swift] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:37.212572 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc67332-8a60-4d13-81e8-898ff2165750 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:37.220411 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc67332-8a60-4d13-81e8-898ff2165750 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:37.221988 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc67332-8a60-4d13-81e8-898ff2165750 service swift] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:37.229834 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 902/1783] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Oct 21 21:22:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 21:22:41.563238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027335bb-ca96-4117-ad4a-d277abc0e90a None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:41.564682 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-027335bb-ca96-4117-ad4a-d277abc0e90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:41.628274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027335bb-ca96-4117-ad4a-d277abc0e90a None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:41.632729 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16084e71-d480-4699-b942-6f490755bd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:41.633350 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16084e71-d480-4699-b942-6f490755bd30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:41.633630 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16084e71-d480-4699-b942-6f490755bd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:41.633904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16084e71-d480-4699-b942-6f490755bd30 admin admin] PATH_INFO: `/v3/users/eb2d67d1a43440a29fbd88704945e27b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:41.635029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16084e71-d480-4699-b942-6f490755bd30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb2d67d1a43440a29fbd88704945e27b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:41.636964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16084e71-d480-4699-b942-6f490755bd30 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:41.682006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-16084e71-d480-4699-b942-6f490755bd30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb2d67d1a43440a29fbd88704945e27b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:41.682655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-16084e71-d480-4699-b942-6f490755bd30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb2d67d1a43440a29fbd88704945e27b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:41.694072 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-16084e71-d480-4699-b942-6f490755bd30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb2d67d1a43440a29fbd88704945e27b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:41.701916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-16084e71-d480-4699-b942-6f490755bd30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb2d67d1a43440a29fbd88704945e27b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:41.702673 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-16084e71-d480-4699-b942-6f490755bd30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb2d67d1a43440a29fbd88704945e27b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:41.714872 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 882/1784] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:22:41 2022] DELETE /identity/v3/users/eb2d67d1a43440a29fbd88704945e27b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:41.719259 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d12d8f-f244-4f10-bcad-15b60809f7fb None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:41.719735 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d12d8f-f244-4f10-bcad-15b60809f7fb None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:41.719824 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d12d8f-f244-4f10-bcad-15b60809f7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:41.739057 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-48d12d8f-f244-4f10-bcad-15b60809f7fb None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:41.740278 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-48d12d8f-f244-4f10-bcad-15b60809f7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:41.800356 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 903/1785] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:41.812874 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16084e71-d480-4699-b942-6f490755bd30 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:41.814476 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-16084e71-d480-4699-b942-6f490755bd30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:41.850596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:41.884937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16084e71-d480-4699-b942-6f490755bd30 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:41.888714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd0fde-282e-4309-acb4-ca1506d4bb8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:41.889347 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd0fde-282e-4309-acb4-ca1506d4bb8a service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:41.889625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd0fde-282e-4309-acb4-ca1506d4bb8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:41.889895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd0fde-282e-4309-acb4-ca1506d4bb8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:41.900667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd0fde-282e-4309-acb4-ca1506d4bb8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:41.902119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd0fde-282e-4309-acb4-ca1506d4bb8a service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:41.913377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 883/1786] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:22:42.160081 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d12d8f-f244-4f10-bcad-15b60809f7fb None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:42.161520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-48d12d8f-f244-4f10-bcad-15b60809f7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:42.173732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d12d8f-f244-4f10-bcad-15b60809f7fb None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:42.178549 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480b3ea-551f-45e3-bfeb-280ec01a21d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:42.179191 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480b3ea-551f-45e3-bfeb-280ec01a21d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:42.179469 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480b3ea-551f-45e3-bfeb-280ec01a21d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:42.179770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480b3ea-551f-45e3-bfeb-280ec01a21d5 admin admin] PATH_INFO: `/v3/projects/4b3cdf29e6cf466ea2bd63de5073aeb5` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:42.180903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480b3ea-551f-45e3-bfeb-280ec01a21d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b3cdf29e6cf466ea2bd63de5073aeb5)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:42.182812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480b3ea-551f-45e3-bfeb-280ec01a21d5 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:42.226565 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:42.229800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-8480b3ea-551f-45e3-bfeb-280ec01a21d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b3cdf29e6cf466ea2bd63de5073aeb5', 'request_id': 'req-8480b3ea-551f-45e3-bfeb-280ec01a21d5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:42.236947 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-8480b3ea-551f-45e3-bfeb-280ec01a21d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b3cdf29e6cf466ea2bd63de5073aeb5', 'request_id': 'req-8480b3ea-551f-45e3-bfeb-280ec01a21d5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:42.243495 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-8480b3ea-551f-45e3-bfeb-280ec01a21d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b3cdf29e6cf466ea2bd63de5073aeb5', 'request_id': 'req-8480b3ea-551f-45e3-bfeb-280ec01a21d5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:42.254068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-8480b3ea-551f-45e3-bfeb-280ec01a21d5 admin admin] The token cache is being invalidate because project 4b3cdf29e6cf466ea2bd63de5073aeb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:22:42.254351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-8480b3ea-551f-45e3-bfeb-280ec01a21d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:42.254966 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-8480b3ea-551f-45e3-bfeb-280ec01a21d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:42.256473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 904/1787] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:22:42 2022] DELETE /identity/v3/projects/4b3cdf29e6cf466ea2bd63de5073aeb5 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:47.183680 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd0fde-282e-4309-acb4-ca1506d4bb8a service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:47.185160 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-72dd0fde-282e-4309-acb4-ca1506d4bb8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:47.241618 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd0fde-282e-4309-acb4-ca1506d4bb8a service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:47.289775 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c4b96-f47b-41bc-aa3e-19a84f26e642 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:47.290298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c4b96-f47b-41bc-aa3e-19a84f26e642 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:47.290575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c4b96-f47b-41bc-aa3e-19a84f26e642 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:47.290839 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c4b96-f47b-41bc-aa3e-19a84f26e642 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:47.292550 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d56c4b96-f47b-41bc-aa3e-19a84f26e642 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:47.353291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c4b96-f47b-41bc-aa3e-19a84f26e642 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:47.354705 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c4b96-f47b-41bc-aa3e-19a84f26e642 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:47.364554 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 884/1788] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:22:52.166962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480b3ea-551f-45e3-bfeb-280ec01a21d5 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:52.168471 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8480b3ea-551f-45e3-bfeb-280ec01a21d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:52.231304 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480b3ea-551f-45e3-bfeb-280ec01a21d5 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:52.236271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0f737-f1e2-4ec4-b005-fa5d5848dacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:52.236900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0f737-f1e2-4ec4-b005-fa5d5848dacf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:52.237175 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0f737-f1e2-4ec4-b005-fa5d5848dacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:52.237439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0f737-f1e2-4ec4-b005-fa5d5848dacf admin admin] PATH_INFO: `/v3/users/3ae046fff4dc4099856df86934b62ca5` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:52.238582 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0f737-f1e2-4ec4-b005-fa5d5848dacf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ae046fff4dc4099856df86934b62ca5)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:52.240528 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0f737-f1e2-4ec4-b005-fa5d5848dacf admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:52.252196 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c4b96-f47b-41bc-aa3e-19a84f26e642 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:52.253715 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d56c4b96-f47b-41bc-aa3e-19a84f26e642 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:52.266906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c4b96-f47b-41bc-aa3e-19a84f26e642 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:52.272686 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c2a3a-b839-4664-a7f0-fe588770f632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:52.273663 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c2a3a-b839-4664-a7f0-fe588770f632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:52.273970 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c2a3a-b839-4664-a7f0-fe588770f632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:52.274304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c2a3a-b839-4664-a7f0-fe588770f632 admin admin] PATH_INFO: `/v3/users/d3e8b787756244bd85449ec4e6018939` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:52.275656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c2a3a-b839-4664-a7f0-fe588770f632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3e8b787756244bd85449ec4e6018939)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:52.277651 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c2a3a-b839-4664-a7f0-fe588770f632 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:52.290347 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9df0f737-f1e2-4ec4-b005-fa5d5848dacf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ae046fff4dc4099856df86934b62ca5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:52.303877 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9df0f737-f1e2-4ec4-b005-fa5d5848dacf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ae046fff4dc4099856df86934b62ca5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:52.308962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:52.311754 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9df0f737-f1e2-4ec4-b005-fa5d5848dacf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ae046fff4dc4099856df86934b62ca5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:52.312456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9df0f737-f1e2-4ec4-b005-fa5d5848dacf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ae046fff4dc4099856df86934b62ca5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:52.320859 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9df0f737-f1e2-4ec4-b005-fa5d5848dacf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ae046fff4dc4099856df86934b62ca5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:52.327111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 905/1789] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:22:52 2022] DELETE /identity/v3/users/3ae046fff4dc4099856df86934b62ca5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:52.327376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c47c2a3a-b839-4664-a7f0-fe588770f632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3e8b787756244bd85449ec4e6018939'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:52.327873 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c47c2a3a-b839-4664-a7f0-fe588770f632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3e8b787756244bd85449ec4e6018939'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:52.331837 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01998142-2eba-4093-bd42-4c6a651f7789 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:52.332124 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01998142-2eba-4093-bd42-4c6a651f7789 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:52.332399 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01998142-2eba-4093-bd42-4c6a651f7789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:52.340649 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c47c2a3a-b839-4664-a7f0-fe588770f632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3e8b787756244bd85449ec4e6018939'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:52.349158 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c47c2a3a-b839-4664-a7f0-fe588770f632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3e8b787756244bd85449ec4e6018939'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:52.349850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c47c2a3a-b839-4664-a7f0-fe588770f632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3e8b787756244bd85449ec4e6018939'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:52.351153 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-01998142-2eba-4093-bd42-4c6a651f7789 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:52.352502 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-01998142-2eba-4093-bd42-4c6a651f7789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:52.362848 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 885/1790] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:22:52 2022] DELETE /identity/v3/users/d3e8b787756244bd85449ec4e6018939 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:22:52.367136 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0a818-f3e5-4e15-9b42-a52efc795576 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:52.367424 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0a818-f3e5-4e15-9b42-a52efc795576 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:52.367738 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0a818-f3e5-4e15-9b42-a52efc795576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:52.386156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-75e0a818-f3e5-4e15-9b42-a52efc795576 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:52.387354 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-75e0a818-f3e5-4e15-9b42-a52efc795576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:52.400502 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:52.415304 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 906/1791] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:22:52.426692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01998142-2eba-4093-bd42-4c6a651f7789 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:52.428480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-01998142-2eba-4093-bd42-4c6a651f7789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:52.465377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 886/1792] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:22:52.472347 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0a818-f3e5-4e15-9b42-a52efc795576 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:52.473979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-75e0a818-f3e5-4e15-9b42-a52efc795576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:52.491531 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01998142-2eba-4093-bd42-4c6a651f7789 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:52.536841 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0a818-f3e5-4e15-9b42-a52efc795576 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:52.540728 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84b546-42ce-4475-a103-59dc15f698f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:52.541382 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84b546-42ce-4475-a103-59dc15f698f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:52.541660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84b546-42ce-4475-a103-59dc15f698f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:52.541985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84b546-42ce-4475-a103-59dc15f698f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:52.552674 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84b546-42ce-4475-a103-59dc15f698f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:52.554390 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84b546-42ce-4475-a103-59dc15f698f3 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:52.555899 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31eced1-0ebb-4581-95da-6922d24151bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:52.556531 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31eced1-0ebb-4581-95da-6922d24151bf service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:52.556863 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31eced1-0ebb-4581-95da-6922d24151bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:52.557144 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31eced1-0ebb-4581-95da-6922d24151bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:52.570478 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 887/1793] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:22:52.571650 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31eced1-0ebb-4581-95da-6922d24151bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:52.573381 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31eced1-0ebb-4581-95da-6922d24151bf service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:52.587905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 907/1794] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:22:53.028575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84b546-42ce-4475-a103-59dc15f698f3 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:53.030170 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-7f84b546-42ce-4475-a103-59dc15f698f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:53.040948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31eced1-0ebb-4581-95da-6922d24151bf service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:53.042559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d31eced1-0ebb-4581-95da-6922d24151bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:53.043410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84b546-42ce-4475-a103-59dc15f698f3 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:53.052585 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15edb5a-88a2-417a-a9b9-e20d00e13127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:53.053384 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15edb5a-88a2-417a-a9b9-e20d00e13127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:53.053623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15edb5a-88a2-417a-a9b9-e20d00e13127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:53.059776 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15edb5a-88a2-417a-a9b9-e20d00e13127 admin admin] PATH_INFO: `/v3/projects/cb9e2b3b4ea14dd7b742ed23fdf2c43c` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:53.060976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31eced1-0ebb-4581-95da-6922d24151bf service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:53.061150 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15edb5a-88a2-417a-a9b9-e20d00e13127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb9e2b3b4ea14dd7b742ed23fdf2c43c)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:53.063054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15edb5a-88a2-417a-a9b9-e20d00e13127 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:53.068210 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca24827-fae1-4ffc-9fc9-90940e7c8724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:53.068933 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca24827-fae1-4ffc-9fc9-90940e7c8724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:53.069283 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca24827-fae1-4ffc-9fc9-90940e7c8724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:53.069646 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca24827-fae1-4ffc-9fc9-90940e7c8724 admin admin] PATH_INFO: `/v3/projects/76a154ba2f174dac88b6478f1eaf41d0` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:53.070994 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca24827-fae1-4ffc-9fc9-90940e7c8724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76a154ba2f174dac88b6478f1eaf41d0)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:53.074160 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca24827-fae1-4ffc-9fc9-90940e7c8724 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:53.114766 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:53.119945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a15edb5a-88a2-417a-a9b9-e20d00e13127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb9e2b3b4ea14dd7b742ed23fdf2c43c', 'request_id': 'req-a15edb5a-88a2-417a-a9b9-e20d00e13127'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.131102 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9ca24827-fae1-4ffc-9fc9-90940e7c8724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76a154ba2f174dac88b6478f1eaf41d0', 'request_id': 'req-9ca24827-fae1-4ffc-9fc9-90940e7c8724'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.131425 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a15edb5a-88a2-417a-a9b9-e20d00e13127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb9e2b3b4ea14dd7b742ed23fdf2c43c', 'request_id': 'req-a15edb5a-88a2-417a-a9b9-e20d00e13127'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.137681 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a15edb5a-88a2-417a-a9b9-e20d00e13127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb9e2b3b4ea14dd7b742ed23fdf2c43c', 'request_id': 'req-a15edb5a-88a2-417a-a9b9-e20d00e13127'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.139100 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9ca24827-fae1-4ffc-9fc9-90940e7c8724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76a154ba2f174dac88b6478f1eaf41d0', 'request_id': 'req-9ca24827-fae1-4ffc-9fc9-90940e7c8724'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.150117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9ca24827-fae1-4ffc-9fc9-90940e7c8724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76a154ba2f174dac88b6478f1eaf41d0', 'request_id': 'req-9ca24827-fae1-4ffc-9fc9-90940e7c8724'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.150453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a15edb5a-88a2-417a-a9b9-e20d00e13127 admin admin] The token cache is being invalidate because project cb9e2b3b4ea14dd7b742ed23fdf2c43c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:22:53.150742 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a15edb5a-88a2-417a-a9b9-e20d00e13127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.151836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a15edb5a-88a2-417a-a9b9-e20d00e13127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.153604 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 888/1795] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:22:53 2022] DELETE /identity/v3/projects/cb9e2b3b4ea14dd7b742ed23fdf2c43c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:53.157681 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15edb5a-88a2-417a-a9b9-e20d00e13127 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:53.159033 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a15edb5a-88a2-417a-a9b9-e20d00e13127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:53.162693 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9ca24827-fae1-4ffc-9fc9-90940e7c8724 admin admin] The token cache is being invalidate because project 76a154ba2f174dac88b6478f1eaf41d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:22:53.162988 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9ca24827-fae1-4ffc-9fc9-90940e7c8724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.163674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9ca24827-fae1-4ffc-9fc9-90940e7c8724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.165459 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 908/1796] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:22:53 2022] DELETE /identity/v3/projects/76a154ba2f174dac88b6478f1eaf41d0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:53.239052 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15edb5a-88a2-417a-a9b9-e20d00e13127 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:53.303631 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8e748-e28c-42f5-91b4-ef680f15354c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:53.304481 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8e748-e28c-42f5-91b4-ef680f15354c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:53.305190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8e748-e28c-42f5-91b4-ef680f15354c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:53.305450 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8e748-e28c-42f5-91b4-ef680f15354c admin admin] PATH_INFO: `/v3/users/87a7096021494a10b681f8c6b4ac734f` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:53.306704 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8e748-e28c-42f5-91b4-ef680f15354c admin admin] RBAC: Authorizing `identity:delete_user(user_id=87a7096021494a10b681f8c6b4ac734f)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:53.308741 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8e748-e28c-42f5-91b4-ef680f15354c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:53.370476 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-7bf8e748-e28c-42f5-91b4-ef680f15354c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87a7096021494a10b681f8c6b4ac734f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.371310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-7bf8e748-e28c-42f5-91b4-ef680f15354c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87a7096021494a10b681f8c6b4ac734f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.384228 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-7bf8e748-e28c-42f5-91b4-ef680f15354c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87a7096021494a10b681f8c6b4ac734f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.392107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-7bf8e748-e28c-42f5-91b4-ef680f15354c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87a7096021494a10b681f8c6b4ac734f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.393230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-7bf8e748-e28c-42f5-91b4-ef680f15354c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87a7096021494a10b681f8c6b4ac734f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.401111 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:53.406895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 889/1797] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:22:53 2022] DELETE /identity/v3/users/87a7096021494a10b681f8c6b4ac734f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:22:53.626505 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca24827-fae1-4ffc-9fc9-90940e7c8724 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:53.628484 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9ca24827-fae1-4ffc-9fc9-90940e7c8724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:53.641047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca24827-fae1-4ffc-9fc9-90940e7c8724 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:53.674944 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:53.682664 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8e748-e28c-42f5-91b4-ef680f15354c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:53.684256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-7bf8e748-e28c-42f5-91b4-ef680f15354c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:53.707157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85437e9-21c9-49c9-80ab-479a20d5c65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:53.707951 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85437e9-21c9-49c9-80ab-479a20d5c65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:53.708230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85437e9-21c9-49c9-80ab-479a20d5c65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:53.708504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85437e9-21c9-49c9-80ab-479a20d5c65a admin admin] PATH_INFO: `/v3/users/17555a4bdd2e424aad4fc9b5ff38d505` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:53.709638 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85437e9-21c9-49c9-80ab-479a20d5c65a admin admin] RBAC: Authorizing `identity:delete_user(user_id=17555a4bdd2e424aad4fc9b5ff38d505)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:53.711796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85437e9-21c9-49c9-80ab-479a20d5c65a admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:53.768550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e85437e9-21c9-49c9-80ab-479a20d5c65a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17555a4bdd2e424aad4fc9b5ff38d505'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.769426 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8e748-e28c-42f5-91b4-ef680f15354c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:53.775268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9126b6-51be-4a47-952a-14f3dc55d474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:53.776024 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9126b6-51be-4a47-952a-14f3dc55d474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:53.776359 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9126b6-51be-4a47-952a-14f3dc55d474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:53.776707 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9126b6-51be-4a47-952a-14f3dc55d474 admin admin] PATH_INFO: `/v3/projects/23889aa3ceee46b3acf80a6610c36c82` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:53.778379 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9126b6-51be-4a47-952a-14f3dc55d474 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23889aa3ceee46b3acf80a6610c36c82)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:53.780452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9126b6-51be-4a47-952a-14f3dc55d474 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:53.784997 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e85437e9-21c9-49c9-80ab-479a20d5c65a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17555a4bdd2e424aad4fc9b5ff38d505'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.794391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e85437e9-21c9-49c9-80ab-479a20d5c65a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17555a4bdd2e424aad4fc9b5ff38d505'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.795368 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e85437e9-21c9-49c9-80ab-479a20d5c65a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17555a4bdd2e424aad4fc9b5ff38d505'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.806449 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e85437e9-21c9-49c9-80ab-479a20d5c65a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17555a4bdd2e424aad4fc9b5ff38d505'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.815337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 909/1798] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:22:53 2022] DELETE /identity/v3/users/17555a4bdd2e424aad4fc9b5ff38d505 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:53.820164 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282df2e5-c13a-4a94-b642-472c535b8cc7 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:53.820681 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282df2e5-c13a-4a94-b642-472c535b8cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:53.821113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282df2e5-c13a-4a94-b642-472c535b8cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:53.832996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ec9126b6-51be-4a47-952a-14f3dc55d474 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23889aa3ceee46b3acf80a6610c36c82', 'request_id': 'req-ec9126b6-51be-4a47-952a-14f3dc55d474'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.844115 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ec9126b6-51be-4a47-952a-14f3dc55d474 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23889aa3ceee46b3acf80a6610c36c82', 'request_id': 'req-ec9126b6-51be-4a47-952a-14f3dc55d474'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.844434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-282df2e5-c13a-4a94-b642-472c535b8cc7 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:53.846279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-282df2e5-c13a-4a94-b642-472c535b8cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:53.852974 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ec9126b6-51be-4a47-952a-14f3dc55d474 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23889aa3ceee46b3acf80a6610c36c82', 'request_id': 'req-ec9126b6-51be-4a47-952a-14f3dc55d474'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.866322 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ec9126b6-51be-4a47-952a-14f3dc55d474 admin admin] The token cache is being invalidate because project 23889aa3ceee46b3acf80a6610c36c82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:22:53.866647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ec9126b6-51be-4a47-952a-14f3dc55d474 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.867494 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ec9126b6-51be-4a47-952a-14f3dc55d474 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:53.873523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 890/1799] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:22:53 2022] DELETE /identity/v3/projects/23889aa3ceee46b3acf80a6610c36c82 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:22:53.874950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ebead-0447-43ac-927b-8a3ecc8476dd None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:53.875237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ebead-0447-43ac-927b-8a3ecc8476dd None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:53.875562 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ebead-0447-43ac-927b-8a3ecc8476dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:53.900160 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-ac2ebead-0447-43ac-927b-8a3ecc8476dd None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:53.901977 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ac2ebead-0447-43ac-927b-8a3ecc8476dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:53.919739 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 910/1800] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:22:53.920315 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282df2e5-c13a-4a94-b642-472c535b8cc7 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:53.921999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-282df2e5-c13a-4a94-b642-472c535b8cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:53.964387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 891/1801] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:22:53.972865 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c308a-2835-428e-a1e5-ffdddbbc5396 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:53.973229 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c308a-2835-428e-a1e5-ffdddbbc5396 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:53.973516 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c308a-2835-428e-a1e5-ffdddbbc5396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:53.982725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282df2e5-c13a-4a94-b642-472c535b8cc7 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:53.987453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7319c-a50e-4abd-810d-e047de009e25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:53.988111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7319c-a50e-4abd-810d-e047de009e25 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:53.988394 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7319c-a50e-4abd-810d-e047de009e25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:53.988672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7319c-a50e-4abd-810d-e047de009e25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:53.992937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-130c308a-2835-428e-a1e5-ffdddbbc5396 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:53.994133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-130c308a-2835-428e-a1e5-ffdddbbc5396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:53.997954 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7319c-a50e-4abd-810d-e047de009e25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:53.999398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7319c-a50e-4abd-810d-e047de009e25 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:54.002215 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 892/1802] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:22:54.003456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:54.008513 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c308a-2835-428e-a1e5-ffdddbbc5396 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:54.010936 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 911/1803] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:22:54.017828 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c308a-2835-428e-a1e5-ffdddbbc5396 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:54.022776 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d42f7-debb-433b-bfde-c05fd3b7b75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:54.023674 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d42f7-debb-433b-bfde-c05fd3b7b75f admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.023956 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d42f7-debb-433b-bfde-c05fd3b7b75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.024226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d42f7-debb-433b-bfde-c05fd3b7b75f admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.024863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d42f7-debb-433b-bfde-c05fd3b7b75f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:22:54.025166 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d42f7-debb-433b-bfde-c05fd3b7b75f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:54.026741 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d42f7-debb-433b-bfde-c05fd3b7b75f admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:54.041948 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 893/1804] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:22:54.043113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7319c-a50e-4abd-810d-e047de009e25 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:54.053077 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7319c-a50e-4abd-810d-e047de009e25 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:54.057707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e475c2b-c37f-43a8-b1c8-f0fa460921ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:54.058496 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e475c2b-c37f-43a8-b1c8-f0fa460921ad admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.058837 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e475c2b-c37f-43a8-b1c8-f0fa460921ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.059173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e475c2b-c37f-43a8-b1c8-f0fa460921ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.059841 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e475c2b-c37f-43a8-b1c8-f0fa460921ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:54.061732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e475c2b-c37f-43a8-b1c8-f0fa460921ad admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:54.094591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 912/1805] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:22:54 2022] POST /identity/v3/projects => generated 359 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:22:54.095718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d42f7-debb-433b-bfde-c05fd3b7b75f admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:54.106109 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d42f7-debb-433b-bfde-c05fd3b7b75f admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:54.112194 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:54.170112 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dec243-e712-4323-9999-2e3e0d68422b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:54.170726 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dec243-e712-4323-9999-2e3e0d68422b admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.171060 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dec243-e712-4323-9999-2e3e0d68422b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.171388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dec243-e712-4323-9999-2e3e0d68422b admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.171940 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dec243-e712-4323-9999-2e3e0d68422b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:54.174411 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dec243-e712-4323-9999-2e3e0d68422b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:54.199818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 894/1806] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:22:54 2022] POST /identity/v3/users => generated 280 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:22:54.200986 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e475c2b-c37f-43a8-b1c8-f0fa460921ad admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:54.211162 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e475c2b-c37f-43a8-b1c8-f0fa460921ad admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:54.215791 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33743bcc-1148-4f0c-bb3a-f62cc78eb7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:54.216394 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33743bcc-1148-4f0c-bb3a-f62cc78eb7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.216667 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33743bcc-1148-4f0c-bb3a-f62cc78eb7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.216933 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33743bcc-1148-4f0c-bb3a-f62cc78eb7a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.217561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33743bcc-1148-4f0c-bb3a-f62cc78eb7a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:54.219153 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33743bcc-1148-4f0c-bb3a-f62cc78eb7a6 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:54.229705 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 913/1807] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:22:54.233818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dec243-e712-4323-9999-2e3e0d68422b admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:54.244253 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dec243-e712-4323-9999-2e3e0d68422b admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:54.248878 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9205d4-60ea-45f0-9393-0430d74d32b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:54.249485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9205d4-60ea-45f0-9393-0430d74d32b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.249768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9205d4-60ea-45f0-9393-0430d74d32b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.250041 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9205d4-60ea-45f0-9393-0430d74d32b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.250678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9205d4-60ea-45f0-9393-0430d74d32b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:54.252948 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9205d4-60ea-45f0-9393-0430d74d32b6 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:54.264528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 895/1808] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:22:54.268853 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33743bcc-1148-4f0c-bb3a-f62cc78eb7a6 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:54.279277 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33743bcc-1148-4f0c-bb3a-f62cc78eb7a6 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:54.284568 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939e14c-f725-47d3-83b6-5e905e4a5fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:54.285224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939e14c-f725-47d3-83b6-5e905e4a5fd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.285504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939e14c-f725-47d3-83b6-5e905e4a5fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.285777 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939e14c-f725-47d3-83b6-5e905e4a5fd9 admin admin] PATH_INFO: `/v3/projects/edf9f416f2de4c65b8cbaef19121c5ba/users/b95001afc831441c87c96087038d117c/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.300537 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939e14c-f725-47d3-83b6-5e905e4a5fd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf9f416f2de4c65b8cbaef19121c5ba, user_id=b95001afc831441c87c96087038d117c, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:54.303204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939e14c-f725-47d3-83b6-5e905e4a5fd9 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:54.308382 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9205d4-60ea-45f0-9393-0430d74d32b6 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:54.310399 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-0b9205d4-60ea-45f0-9393-0430d74d32b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:54.320157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 914/1809] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:22:54 2022] PUT /identity/v3/projects/edf9f416f2de4c65b8cbaef19121c5ba/users/b95001afc831441c87c96087038d117c/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:54.324467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9205d4-60ea-45f0-9393-0430d74d32b6 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:54.330997 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce39dc-0d4b-41ba-9bb2-b42ed67735ed None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.331315 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce39dc-0d4b-41ba-9bb2-b42ed67735ed None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.331615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce39dc-0d4b-41ba-9bb2-b42ed67735ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.362318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-bfce39dc-0d4b-41ba-9bb2-b42ed67735ed None None] MFA Rules not processed for user `b95001afc831441c87c96087038d117c`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:54.363520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-bfce39dc-0d4b-41ba-9bb2-b42ed67735ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:54.394944 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8944f-f286-43b0-9dd2-d9a61bd2861b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:54.395960 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8944f-f286-43b0-9dd2-d9a61bd2861b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.396387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8944f-f286-43b0-9dd2-d9a61bd2861b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.396665 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8944f-f286-43b0-9dd2-d9a61bd2861b admin admin] PATH_INFO: `/v3/projects/9ab6184daf7148fa88092d98631e1f0e` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.397778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8944f-f286-43b0-9dd2-d9a61bd2861b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ab6184daf7148fa88092d98631e1f0e)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:54.400013 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8944f-f286-43b0-9dd2-d9a61bd2861b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:54.431112 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 915/1810] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:54 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:22:54.447438 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bad8944f-f286-43b0-9dd2-d9a61bd2861b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ab6184daf7148fa88092d98631e1f0e', 'request_id': 'req-bad8944f-f286-43b0-9dd2-d9a61bd2861b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:54.456603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bad8944f-f286-43b0-9dd2-d9a61bd2861b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ab6184daf7148fa88092d98631e1f0e', 'request_id': 'req-bad8944f-f286-43b0-9dd2-d9a61bd2861b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:54.462711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bad8944f-f286-43b0-9dd2-d9a61bd2861b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ab6184daf7148fa88092d98631e1f0e', 'request_id': 'req-bad8944f-f286-43b0-9dd2-d9a61bd2861b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:54.467938 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce39dc-0d4b-41ba-9bb2-b42ed67735ed None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:54.469383 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-bfce39dc-0d4b-41ba-9bb2-b42ed67735ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:54.474243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bad8944f-f286-43b0-9dd2-d9a61bd2861b admin admin] The token cache is being invalidate because project 9ab6184daf7148fa88092d98631e1f0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:22:54.474531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bad8944f-f286-43b0-9dd2-d9a61bd2861b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:54.475334 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-bad8944f-f286-43b0-9dd2-d9a61bd2861b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:54.477032 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 896/1811] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:22:54 2022] DELETE /identity/v3/projects/9ab6184daf7148fa88092d98631e1f0e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:22:54.481726 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41195368-74c4-4285-baf4-28d0c8816ccf None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.482012 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41195368-74c4-4285-baf4-28d0c8816ccf None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.482286 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41195368-74c4-4285-baf4-28d0c8816ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.501979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-41195368-74c4-4285-baf4-28d0c8816ccf None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:54.503164 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-41195368-74c4-4285-baf4-28d0c8816ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:54.516836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:54.528134 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce39dc-0d4b-41ba-9bb2-b42ed67735ed None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:54.569641 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 897/1812] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:22:54.573086 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:54.577689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f777fb-d022-4eb1-876c-75b56a989b2d None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.577982 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f777fb-d022-4eb1-876c-75b56a989b2d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.578301 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f777fb-d022-4eb1-876c-75b56a989b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.581167 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37120a8b-5c89-49a0-9782-474a8590aabb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:54.582039 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37120a8b-5c89-49a0-9782-474a8590aabb service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.582665 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37120a8b-5c89-49a0-9782-474a8590aabb service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.582943 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37120a8b-5c89-49a0-9782-474a8590aabb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.584712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-37120a8b-5c89-49a0-9782-474a8590aabb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:54.598309 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-35f777fb-d022-4eb1-876c-75b56a989b2d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:54.599503 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-35f777fb-d022-4eb1-876c-75b56a989b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:54.607451 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 898/1813] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:22:54.612162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f777fb-d022-4eb1-876c-75b56a989b2d None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:54.620952 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f777fb-d022-4eb1-876c-75b56a989b2d None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:54.625577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0d299-f49d-4978-8b77-61543fc3e12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:54.626163 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0d299-f49d-4978-8b77-61543fc3e12b admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.626504 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0d299-f49d-4978-8b77-61543fc3e12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.626778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0d299-f49d-4978-8b77-61543fc3e12b admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.627413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0d299-f49d-4978-8b77-61543fc3e12b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:22:54.627747 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0d299-f49d-4978-8b77-61543fc3e12b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:54.629362 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0d299-f49d-4978-8b77-61543fc3e12b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:54.640919 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 899/1814] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:22:54.645413 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37120a8b-5c89-49a0-9782-474a8590aabb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:54.645892 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0d299-f49d-4978-8b77-61543fc3e12b admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:54.646804 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37120a8b-5c89-49a0-9782-474a8590aabb service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:54.655312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0d299-f49d-4978-8b77-61543fc3e12b admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:54.656369 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 916/1815] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:54 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:22:54.660974 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7fed9-259b-4f3e-9901-22815490d9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:54.661562 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7fed9-259b-4f3e-9901-22815490d9e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.661867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7fed9-259b-4f3e-9901-22815490d9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.662149 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7fed9-259b-4f3e-9901-22815490d9e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.662845 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7fed9-259b-4f3e-9901-22815490d9e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:54.664944 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7fed9-259b-4f3e-9901-22815490d9e9 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:54.672570 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37120a8b-5c89-49a0-9782-474a8590aabb service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:54.673983 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-37120a8b-5c89-49a0-9782-474a8590aabb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:54.688803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37120a8b-5c89-49a0-9782-474a8590aabb service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:54.693636 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c1c6f-7eae-4d57-b9c5-49dca80ba128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:54.694263 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c1c6f-7eae-4d57-b9c5-49dca80ba128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.694542 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c1c6f-7eae-4d57-b9c5-49dca80ba128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.694817 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c1c6f-7eae-4d57-b9c5-49dca80ba128 admin admin] PATH_INFO: `/v3/users/b95001afc831441c87c96087038d117c` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.698648 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 900/1816] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:22:54 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:22:54.698782 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c1c6f-7eae-4d57-b9c5-49dca80ba128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b95001afc831441c87c96087038d117c)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:54.698782 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c1c6f-7eae-4d57-b9c5-49dca80ba128 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:54.699762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7fed9-259b-4f3e-9901-22815490d9e9 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:54.752133 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e06c1c6f-7eae-4d57-b9c5-49dca80ba128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b95001afc831441c87c96087038d117c'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:54.764298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7fed9-259b-4f3e-9901-22815490d9e9 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:54.766622 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e06c1c6f-7eae-4d57-b9c5-49dca80ba128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b95001afc831441c87c96087038d117c'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:54.769420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f3a82-cd48-4b4c-a874-1eccd8021297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:54.769736 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f3a82-cd48-4b4c-a874-1eccd8021297 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.770019 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f3a82-cd48-4b4c-a874-1eccd8021297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.770294 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f3a82-cd48-4b4c-a874-1eccd8021297 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.771117 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f3a82-cd48-4b4c-a874-1eccd8021297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:54.773173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f3a82-cd48-4b4c-a874-1eccd8021297 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:54.775394 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e06c1c6f-7eae-4d57-b9c5-49dca80ba128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b95001afc831441c87c96087038d117c'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:54.776023 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e06c1c6f-7eae-4d57-b9c5-49dca80ba128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b95001afc831441c87c96087038d117c'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:54.783495 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e06c1c6f-7eae-4d57-b9c5-49dca80ba128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b95001afc831441c87c96087038d117c'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:54.790667 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 917/1817] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:22:54 2022] DELETE /identity/v3/users/b95001afc831441c87c96087038d117c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:22:54.794972 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf7853-9ad6-4790-bf85-db00039ba8c5 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.795286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf7853-9ad6-4790-bf85-db00039ba8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.795538 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf7853-9ad6-4790-bf85-db00039ba8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.798352 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 901/1818] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:22:54 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:22:54.802757 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f3a82-cd48-4b4c-a874-1eccd8021297 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:54.804373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-718f3a82-cd48-4b4c-a874-1eccd8021297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:54.814791 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-9bcf7853-9ad6-4790-bf85-db00039ba8c5 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:54.816001 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9bcf7853-9ad6-4790-bf85-db00039ba8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:54.816482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f3a82-cd48-4b4c-a874-1eccd8021297 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:54.878030 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577dedac-42ee-4086-975b-4fa3103c9460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:54.878517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 918/1819] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:22:54.878628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577dedac-42ee-4086-975b-4fa3103c9460 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.878914 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577dedac-42ee-4086-975b-4fa3103c9460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.879275 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577dedac-42ee-4086-975b-4fa3103c9460 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.880027 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577dedac-42ee-4086-975b-4fa3103c9460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:54.881759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577dedac-42ee-4086-975b-4fa3103c9460 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:54.888602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf7853-9ad6-4790-bf85-db00039ba8c5 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:54.889985 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9bcf7853-9ad6-4790-bf85-db00039ba8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:54.892031 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 902/1820] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:22:54.896248 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577dedac-42ee-4086-975b-4fa3103c9460 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:54.901759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:54.905225 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:54.908244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577dedac-42ee-4086-975b-4fa3103c9460 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:54.913766 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf9440-7738-40b5-b056-3ad9d75a4d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:54.914436 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf9440-7738-40b5-b056-3ad9d75a4d2f admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.914717 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf9440-7738-40b5-b056-3ad9d75a4d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.914994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf9440-7738-40b5-b056-3ad9d75a4d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.915650 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf9440-7738-40b5-b056-3ad9d75a4d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:54.917234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf9440-7738-40b5-b056-3ad9d75a4d2f admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:54.930541 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 903/1821] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:22:54.931997 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf9440-7738-40b5-b056-3ad9d75a4d2f admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:54.941739 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf9440-7738-40b5-b056-3ad9d75a4d2f admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:54.947081 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7966102-0fd9-4dba-9962-1a6c38173822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:54.947938 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7966102-0fd9-4dba-9962-1a6c38173822 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.948375 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7966102-0fd9-4dba-9962-1a6c38173822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.948647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7966102-0fd9-4dba-9962-1a6c38173822 admin admin] PATH_INFO: `/v3/projects/b2ab91f2df9943bc9c607d69660af3b2/users/e6ba3286fd7144899632252abb9549ac/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.953068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf7853-9ad6-4790-bf85-db00039ba8c5 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:54.957337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0f07c-2a66-4f95-9c5e-e850554f1f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:54.957945 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0f07c-2a66-4f95-9c5e-e850554f1f3b service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.958231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0f07c-2a66-4f95-9c5e-e850554f1f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.958506 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0f07c-2a66-4f95-9c5e-e850554f1f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:54.964987 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7966102-0fd9-4dba-9962-1a6c38173822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2ab91f2df9943bc9c607d69660af3b2, user_id=e6ba3286fd7144899632252abb9549ac, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:54.967658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7966102-0fd9-4dba-9962-1a6c38173822 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:54.968650 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0f07c-2a66-4f95-9c5e-e850554f1f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:54.970124 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0f07c-2a66-4f95-9c5e-e850554f1f3b service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:54.980517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 919/1822] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:22:54.980891 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 904/1823] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:22:54 2022] PUT /identity/v3/projects/b2ab91f2df9943bc9c607d69660af3b2/users/e6ba3286fd7144899632252abb9549ac/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:54.992358 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714cb22-547b-4022-b30f-7f0fcd46e2b5 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:54.992647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714cb22-547b-4022-b30f-7f0fcd46e2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:54.992921 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714cb22-547b-4022-b30f-7f0fcd46e2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:55.032598 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-f714cb22-547b-4022-b30f-7f0fcd46e2b5 None None] MFA Rules not processed for user `e6ba3286fd7144899632252abb9549ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:55.033788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f714cb22-547b-4022-b30f-7f0fcd46e2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:55.118419 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 920/1824] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:54 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:55.132205 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7966102-0fd9-4dba-9962-1a6c38173822 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:55.134035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-b7966102-0fd9-4dba-9962-1a6c38173822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:55.199503 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7966102-0fd9-4dba-9962-1a6c38173822 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:55.203239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78e813-c98f-4df3-8824-be223c293208 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:55.203952 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78e813-c98f-4df3-8824-be223c293208 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:55.204292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78e813-c98f-4df3-8824-be223c293208 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:55.205151 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78e813-c98f-4df3-8824-be223c293208 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:55.213439 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78e813-c98f-4df3-8824-be223c293208 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:55.214849 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78e813-c98f-4df3-8824-be223c293208 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:55.224365 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 905/1825] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:55 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:22:55.277805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714cb22-547b-4022-b30f-7f0fcd46e2b5 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:55.279286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f714cb22-547b-4022-b30f-7f0fcd46e2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:55.351535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714cb22-547b-4022-b30f-7f0fcd46e2b5 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:55.356633 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb70760-bcfa-403c-b650-3eb7fd414a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:55.357456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb70760-bcfa-403c-b650-3eb7fd414a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:55.357737 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb70760-bcfa-403c-b650-3eb7fd414a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:55.358014 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb70760-bcfa-403c-b650-3eb7fd414a24 admin admin] PATH_INFO: `/v3/projects/edf9f416f2de4c65b8cbaef19121c5ba` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:55.359477 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb70760-bcfa-403c-b650-3eb7fd414a24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edf9f416f2de4c65b8cbaef19121c5ba)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:55.362170 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb70760-bcfa-403c-b650-3eb7fd414a24 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:55.410535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:55.412795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0cb70760-bcfa-403c-b650-3eb7fd414a24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edf9f416f2de4c65b8cbaef19121c5ba', 'request_id': 'req-0cb70760-bcfa-403c-b650-3eb7fd414a24'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:55.420470 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0cb70760-bcfa-403c-b650-3eb7fd414a24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edf9f416f2de4c65b8cbaef19121c5ba', 'request_id': 'req-0cb70760-bcfa-403c-b650-3eb7fd414a24'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:55.428657 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0cb70760-bcfa-403c-b650-3eb7fd414a24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edf9f416f2de4c65b8cbaef19121c5ba', 'request_id': 'req-0cb70760-bcfa-403c-b650-3eb7fd414a24'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:55.439673 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0cb70760-bcfa-403c-b650-3eb7fd414a24 admin admin] The token cache is being invalidate because project edf9f416f2de4c65b8cbaef19121c5ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:22:55.439960 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0cb70760-bcfa-403c-b650-3eb7fd414a24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:55.440588 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0cb70760-bcfa-403c-b650-3eb7fd414a24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:55.442091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 921/1826] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:22:55 2022] DELETE /identity/v3/projects/edf9f416f2de4c65b8cbaef19121c5ba => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:55.447164 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330af13-ac3e-4939-9c2c-4dc0a38afc10 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:55.447635 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330af13-ac3e-4939-9c2c-4dc0a38afc10 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:55.447734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330af13-ac3e-4939-9c2c-4dc0a38afc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:55.468333 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-c330af13-ac3e-4939-9c2c-4dc0a38afc10 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:55.469535 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c330af13-ac3e-4939-9c2c-4dc0a38afc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:55.531373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 906/1827] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:22:55.539748 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f489997-caf3-4bfb-b80f-32dd4bc5be85 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:55.540055 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f489997-caf3-4bfb-b80f-32dd4bc5be85 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:55.540311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f489997-caf3-4bfb-b80f-32dd4bc5be85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:55.560002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-2f489997-caf3-4bfb-b80f-32dd4bc5be85 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:55.561247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2f489997-caf3-4bfb-b80f-32dd4bc5be85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:55.570614 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 922/1828] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:55.575036 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330af13-ac3e-4939-9c2c-4dc0a38afc10 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:55.586265 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330af13-ac3e-4939-9c2c-4dc0a38afc10 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:55.592189 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452a762-3ac4-46a7-95ec-f14be139f89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:55.593739 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452a762-3ac4-46a7-95ec-f14be139f89d admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:55.594221 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452a762-3ac4-46a7-95ec-f14be139f89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:55.594750 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452a762-3ac4-46a7-95ec-f14be139f89d admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:55.596219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452a762-3ac4-46a7-95ec-f14be139f89d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:22:55.596706 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452a762-3ac4-46a7-95ec-f14be139f89d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:55.598506 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452a762-3ac4-46a7-95ec-f14be139f89d admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:55.613806 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 907/1829] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:22:55.618380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f489997-caf3-4bfb-b80f-32dd4bc5be85 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:55.628752 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f489997-caf3-4bfb-b80f-32dd4bc5be85 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:55.634018 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d49824-b717-44e6-98ad-d2325fb5b974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:55.634686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d49824-b717-44e6-98ad-d2325fb5b974 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:55.635054 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d49824-b717-44e6-98ad-d2325fb5b974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:55.635314 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d49824-b717-44e6-98ad-d2325fb5b974 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:55.636028 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d49824-b717-44e6-98ad-d2325fb5b974 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:55.638120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d49824-b717-44e6-98ad-d2325fb5b974 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:55.679248 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 923/1830] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:22:55 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:22:55.683828 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452a762-3ac4-46a7-95ec-f14be139f89d admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:55.693942 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452a762-3ac4-46a7-95ec-f14be139f89d admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:55.744358 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:55.751158 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985d308-144d-47a5-a919-0e8fca522c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:55.751915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985d308-144d-47a5-a919-0e8fca522c5b admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:55.752197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985d308-144d-47a5-a919-0e8fca522c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:55.752473 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985d308-144d-47a5-a919-0e8fca522c5b admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:55.753105 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985d308-144d-47a5-a919-0e8fca522c5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:55.755220 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985d308-144d-47a5-a919-0e8fca522c5b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:55.775695 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 908/1831] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:22:55 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:22:55.779787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d49824-b717-44e6-98ad-d2325fb5b974 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:55.790711 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d49824-b717-44e6-98ad-d2325fb5b974 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:55.795286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43f19f-ecbf-4986-9eb6-b88341e79fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:55.795915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43f19f-ecbf-4986-9eb6-b88341e79fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:55.796191 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43f19f-ecbf-4986-9eb6-b88341e79fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:55.796459 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43f19f-ecbf-4986-9eb6-b88341e79fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:55.797083 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43f19f-ecbf-4986-9eb6-b88341e79fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:55.798749 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b43f19f-ecbf-4986-9eb6-b88341e79fb1 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:55.808351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 924/1832] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:22:55.812231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985d308-144d-47a5-a919-0e8fca522c5b admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:55.822531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985d308-144d-47a5-a919-0e8fca522c5b admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:55.827114 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde17417-6e2c-4a83-8110-e6c7ec4a59ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:55.827733 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde17417-6e2c-4a83-8110-e6c7ec4a59ee admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:55.828014 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde17417-6e2c-4a83-8110-e6c7ec4a59ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:55.828283 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde17417-6e2c-4a83-8110-e6c7ec4a59ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:55.828905 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde17417-6e2c-4a83-8110-e6c7ec4a59ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:55.831098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde17417-6e2c-4a83-8110-e6c7ec4a59ee admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:55.841504 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 909/1833] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:22:55.845641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43f19f-ecbf-4986-9eb6-b88341e79fb1 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:55.855453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43f19f-ecbf-4986-9eb6-b88341e79fb1 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:55.860531 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7b50f-964c-4d49-8a80-779334e14aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:55.861714 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7b50f-964c-4d49-8a80-779334e14aec admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:55.861986 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7b50f-964c-4d49-8a80-779334e14aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:55.862257 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7b50f-964c-4d49-8a80-779334e14aec admin admin] PATH_INFO: `/v3/projects/642efe91e0934f51922c52048cddb718/users/88f0ed52bc2f4e6e967c385f286daaf3/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:55.877581 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7b50f-964c-4d49-8a80-779334e14aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=642efe91e0934f51922c52048cddb718, user_id=88f0ed52bc2f4e6e967c385f286daaf3, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:55.880291 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7b50f-964c-4d49-8a80-779334e14aec admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:55.894502 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 925/1834] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:22:55 2022] PUT /identity/v3/projects/642efe91e0934f51922c52048cddb718/users/88f0ed52bc2f4e6e967c385f286daaf3/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:55.905935 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174ffeb-06d3-45cb-8577-736bc3f02103 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:55.906220 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174ffeb-06d3-45cb-8577-736bc3f02103 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:55.906498 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174ffeb-06d3-45cb-8577-736bc3f02103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:55.934829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-8174ffeb-06d3-45cb-8577-736bc3f02103 None None] MFA Rules not processed for user `88f0ed52bc2f4e6e967c385f286daaf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:55.936073 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8174ffeb-06d3-45cb-8577-736bc3f02103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:56.000775 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 910/1835] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:56.012768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7b50f-964c-4d49-8a80-779334e14aec admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:56.014466 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0bf7b50f-964c-4d49-8a80-779334e14aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:56.071390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7b50f-964c-4d49-8a80-779334e14aec admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:56.075116 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ac0ec-66a4-4c8e-ba09-0de055595f92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:56.075765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ac0ec-66a4-4c8e-ba09-0de055595f92 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:56.076046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ac0ec-66a4-4c8e-ba09-0de055595f92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:56.076320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ac0ec-66a4-4c8e-ba09-0de055595f92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:56.086173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ac0ec-66a4-4c8e-ba09-0de055595f92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:56.087634 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ac0ec-66a4-4c8e-ba09-0de055595f92 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:56.097294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 926/1836] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:22:56.121593 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174ffeb-06d3-45cb-8577-736bc3f02103 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:56.195230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174ffeb-06d3-45cb-8577-736bc3f02103 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:56.199907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c87aa1-d9cb-46cf-9e14-93e0e368cf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:56.200543 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c87aa1-d9cb-46cf-9e14-93e0e368cf04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:56.200822 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c87aa1-d9cb-46cf-9e14-93e0e368cf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:56.201144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c87aa1-d9cb-46cf-9e14-93e0e368cf04 admin admin] PATH_INFO: `/v3/users/88f0ed52bc2f4e6e967c385f286daaf3` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:56.202284 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c87aa1-d9cb-46cf-9e14-93e0e368cf04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88f0ed52bc2f4e6e967c385f286daaf3)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:56.204227 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c87aa1-d9cb-46cf-9e14-93e0e368cf04 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:56.275017 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-04c87aa1-d9cb-46cf-9e14-93e0e368cf04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88f0ed52bc2f4e6e967c385f286daaf3'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:56.275761 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-04c87aa1-d9cb-46cf-9e14-93e0e368cf04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88f0ed52bc2f4e6e967c385f286daaf3'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:56.292172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-04c87aa1-d9cb-46cf-9e14-93e0e368cf04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88f0ed52bc2f4e6e967c385f286daaf3'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:56.295732 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:56.305772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-04c87aa1-d9cb-46cf-9e14-93e0e368cf04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88f0ed52bc2f4e6e967c385f286daaf3'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:56.306876 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-04c87aa1-d9cb-46cf-9e14-93e0e368cf04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88f0ed52bc2f4e6e967c385f286daaf3'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:56.320616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 911/1837] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:22:56 2022] DELETE /identity/v3/users/88f0ed52bc2f4e6e967c385f286daaf3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:56.325062 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb4c4f-0fbe-47dc-9754-3ab82b4f57ef None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:56.325345 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb4c4f-0fbe-47dc-9754-3ab82b4f57ef None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:56.325626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb4c4f-0fbe-47dc-9754-3ab82b4f57ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:56.344451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-b9eb4c4f-0fbe-47dc-9754-3ab82b4f57ef None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:56.346299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b9eb4c4f-0fbe-47dc-9754-3ab82b4f57ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:56.408912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 927/1838] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:56.431695 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c87aa1-d9cb-46cf-9e14-93e0e368cf04 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:56.433223 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-04c87aa1-d9cb-46cf-9e14-93e0e368cf04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:56.494741 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c87aa1-d9cb-46cf-9e14-93e0e368cf04 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:56.498525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5a7fd-b5e2-4682-8919-33a07bc06f22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:56.499136 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5a7fd-b5e2-4682-8919-33a07bc06f22 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:56.499412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5a7fd-b5e2-4682-8919-33a07bc06f22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:56.499716 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5a7fd-b5e2-4682-8919-33a07bc06f22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:56.509756 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5a7fd-b5e2-4682-8919-33a07bc06f22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:56.511173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5a7fd-b5e2-4682-8919-33a07bc06f22 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:56.522760 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 912/1839] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:22:56.864656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb4c4f-0fbe-47dc-9754-3ab82b4f57ef None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:56.866273 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b9eb4c4f-0fbe-47dc-9754-3ab82b4f57ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:56.904056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb4c4f-0fbe-47dc-9754-3ab82b4f57ef None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:56.909768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd96fa8-0c70-4086-86d0-21c09a7e7df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:56.910410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd96fa8-0c70-4086-86d0-21c09a7e7df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:56.910841 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd96fa8-0c70-4086-86d0-21c09a7e7df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:56.911111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd96fa8-0c70-4086-86d0-21c09a7e7df4 admin admin] PATH_INFO: `/v3/projects/642efe91e0934f51922c52048cddb718` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:56.912268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd96fa8-0c70-4086-86d0-21c09a7e7df4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=642efe91e0934f51922c52048cddb718)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:56.914181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd96fa8-0c70-4086-86d0-21c09a7e7df4 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:56.926370 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:56.963882 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0dd96fa8-0c70-4086-86d0-21c09a7e7df4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '642efe91e0934f51922c52048cddb718', 'request_id': 'req-0dd96fa8-0c70-4086-86d0-21c09a7e7df4'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:56.970481 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0dd96fa8-0c70-4086-86d0-21c09a7e7df4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '642efe91e0934f51922c52048cddb718', 'request_id': 'req-0dd96fa8-0c70-4086-86d0-21c09a7e7df4'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:56.977100 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0dd96fa8-0c70-4086-86d0-21c09a7e7df4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '642efe91e0934f51922c52048cddb718', 'request_id': 'req-0dd96fa8-0c70-4086-86d0-21c09a7e7df4'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:56.988227 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0dd96fa8-0c70-4086-86d0-21c09a7e7df4 admin admin] The token cache is being invalidate because project 642efe91e0934f51922c52048cddb718 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:22:56.988508 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0dd96fa8-0c70-4086-86d0-21c09a7e7df4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:56.989122 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0dd96fa8-0c70-4086-86d0-21c09a7e7df4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:22:56.990592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 928/1840] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:22:56 2022] DELETE /identity/v3/projects/642efe91e0934f51922c52048cddb718 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:56.995373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b35153-bd05-4456-9a66-4d7ea979f48d None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:56.995624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b35153-bd05-4456-9a66-4d7ea979f48d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:56.995899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b35153-bd05-4456-9a66-4d7ea979f48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:57.015874 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-82b35153-bd05-4456-9a66-4d7ea979f48d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:57.017052 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-82b35153-bd05-4456-9a66-4d7ea979f48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:57.074404 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 913/1841] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:57.082214 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03584edd-7085-483b-a74d-ac73cb9656b9 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:57.082505 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03584edd-7085-483b-a74d-ac73cb9656b9 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:57.082805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03584edd-7085-483b-a74d-ac73cb9656b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:57.104607 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-03584edd-7085-483b-a74d-ac73cb9656b9 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:57.105836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-03584edd-7085-483b-a74d-ac73cb9656b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:57.114205 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 929/1842] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:22:57.118432 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b35153-bd05-4456-9a66-4d7ea979f48d None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:57.128511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b35153-bd05-4456-9a66-4d7ea979f48d None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:57.132991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fc94b-ee04-44cb-81fc-571361b861d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:57.133586 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fc94b-ee04-44cb-81fc-571361b861d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:57.133865 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fc94b-ee04-44cb-81fc-571361b861d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:57.134136 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fc94b-ee04-44cb-81fc-571361b861d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:57.134772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fc94b-ee04-44cb-81fc-571361b861d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:22:57.135075 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fc94b-ee04-44cb-81fc-571361b861d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:57.136680 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fc94b-ee04-44cb-81fc-571361b861d7 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:57.148767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 914/1843] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:22:57.152972 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03584edd-7085-483b-a74d-ac73cb9656b9 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:57.162250 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03584edd-7085-483b-a74d-ac73cb9656b9 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:57.167077 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33421cd-da18-42d5-8ec4-37a589ddaba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:57.167713 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33421cd-da18-42d5-8ec4-37a589ddaba6 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:57.167992 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33421cd-da18-42d5-8ec4-37a589ddaba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:57.168263 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33421cd-da18-42d5-8ec4-37a589ddaba6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:57.168924 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33421cd-da18-42d5-8ec4-37a589ddaba6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:57.170888 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33421cd-da18-42d5-8ec4-37a589ddaba6 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:57.200562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 930/1844] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:22:57 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:22:57.201853 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fc94b-ee04-44cb-81fc-571361b861d7 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:57.211768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fc94b-ee04-44cb-81fc-571361b861d7 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:57.267213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed978f-52b5-4054-8746-07df3c640e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:57.267789 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed978f-52b5-4054-8746-07df3c640e4c admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:57.268069 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed978f-52b5-4054-8746-07df3c640e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:57.268337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed978f-52b5-4054-8746-07df3c640e4c admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:57.268960 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed978f-52b5-4054-8746-07df3c640e4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:57.270844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed978f-52b5-4054-8746-07df3c640e4c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:57.291690 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 915/1845] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:22:57 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:22:57.295968 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33421cd-da18-42d5-8ec4-37a589ddaba6 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:57.306593 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33421cd-da18-42d5-8ec4-37a589ddaba6 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:57.311056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b813b-7320-4f81-ac9c-423f3c198653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:57.311674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b813b-7320-4f81-ac9c-423f3c198653 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:57.311950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b813b-7320-4f81-ac9c-423f3c198653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:57.312215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b813b-7320-4f81-ac9c-423f3c198653 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:57.312839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b813b-7320-4f81-ac9c-423f3c198653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:57.314461 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b813b-7320-4f81-ac9c-423f3c198653 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:57.324006 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 931/1846] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:22:57.327807 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed978f-52b5-4054-8746-07df3c640e4c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:57.334494 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:22:57.338561 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed978f-52b5-4054-8746-07df3c640e4c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:57.344592 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61806935-32f4-4511-82c7-d2beafea1a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:57.345486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61806935-32f4-4511-82c7-d2beafea1a37 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:57.345927 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61806935-32f4-4511-82c7-d2beafea1a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:57.346207 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61806935-32f4-4511-82c7-d2beafea1a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:57.346877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61806935-32f4-4511-82c7-d2beafea1a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:57.348697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61806935-32f4-4511-82c7-d2beafea1a37 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:57.359137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 916/1847] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:22:57.364002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b813b-7320-4f81-ac9c-423f3c198653 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:57.373751 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b813b-7320-4f81-ac9c-423f3c198653 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:57.380395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446a92d-1f95-4dd4-b01c-58795d1266c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:57.381072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446a92d-1f95-4dd4-b01c-58795d1266c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:57.381354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446a92d-1f95-4dd4-b01c-58795d1266c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:57.381628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446a92d-1f95-4dd4-b01c-58795d1266c6 admin admin] PATH_INFO: `/v3/projects/75517fe5629c4f388aca45e08e49ae89/users/1132c97a102b408c8aaeca4ab9393bb4/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:57.396436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446a92d-1f95-4dd4-b01c-58795d1266c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75517fe5629c4f388aca45e08e49ae89, user_id=1132c97a102b408c8aaeca4ab9393bb4, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:57.399357 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446a92d-1f95-4dd4-b01c-58795d1266c6 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:57.412197 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 932/1848] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:22:57 2022] PUT /identity/v3/projects/75517fe5629c4f388aca45e08e49ae89/users/1132c97a102b408c8aaeca4ab9393bb4/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:22:57.424177 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689c277-90cf-4c47-b1aa-e25d2ff3a35a None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:57.424458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689c277-90cf-4c47-b1aa-e25d2ff3a35a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:57.424726 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689c277-90cf-4c47-b1aa-e25d2ff3a35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:57.452857 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-2689c277-90cf-4c47-b1aa-e25d2ff3a35a None None] MFA Rules not processed for user `1132c97a102b408c8aaeca4ab9393bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:22:57.454075 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2689c277-90cf-4c47-b1aa-e25d2ff3a35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:57.523280 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 917/1849] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:22:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:22:57.534664 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446a92d-1f95-4dd4-b01c-58795d1266c6 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:22:57.536437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-6446a92d-1f95-4dd4-b01c-58795d1266c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:22:57.593068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446a92d-1f95-4dd4-b01c-58795d1266c6 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:22:57.596635 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aedbb1b-b0d5-4b74-bcf6-9622845388ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:22:57.597261 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedbb1b-b0d5-4b74-bcf6-9622845388ae service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:22:57.597533 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedbb1b-b0d5-4b74-bcf6-9622845388ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:22:57.597799 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedbb1b-b0d5-4b74-bcf6-9622845388ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:22:57.605696 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aedbb1b-b0d5-4b74-bcf6-9622845388ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:22:57.607184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aedbb1b-b0d5-4b74-bcf6-9622845388ae service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:22:57.616553 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 933/1850] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:22:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:23:04.370544 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689c277-90cf-4c47-b1aa-e25d2ff3a35a None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:04.372020 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2689c277-90cf-4c47-b1aa-e25d2ff3a35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:04.436560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689c277-90cf-4c47-b1aa-e25d2ff3a35a None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:04.441674 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26825bf-fbc3-42ca-9b08-d262948bd78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:04.442300 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26825bf-fbc3-42ca-9b08-d262948bd78f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:04.442580 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26825bf-fbc3-42ca-9b08-d262948bd78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:04.442853 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26825bf-fbc3-42ca-9b08-d262948bd78f admin admin] PATH_INFO: `/v3/users/e6ba3286fd7144899632252abb9549ac` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:04.444006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26825bf-fbc3-42ca-9b08-d262948bd78f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6ba3286fd7144899632252abb9549ac)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:04.446190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26825bf-fbc3-42ca-9b08-d262948bd78f admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:04.492736 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f26825bf-fbc3-42ca-9b08-d262948bd78f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6ba3286fd7144899632252abb9549ac'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:04.492880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f26825bf-fbc3-42ca-9b08-d262948bd78f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6ba3286fd7144899632252abb9549ac'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:04.505564 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f26825bf-fbc3-42ca-9b08-d262948bd78f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6ba3286fd7144899632252abb9549ac'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:04.513419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f26825bf-fbc3-42ca-9b08-d262948bd78f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6ba3286fd7144899632252abb9549ac'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:04.514264 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f26825bf-fbc3-42ca-9b08-d262948bd78f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6ba3286fd7144899632252abb9549ac'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:04.530981 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 918/1851] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:23:04 2022] DELETE /identity/v3/users/e6ba3286fd7144899632252abb9549ac => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:23:04.533414 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369662ed-2ee9-406a-83c1-f16dbd6c0e84 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:04.533870 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369662ed-2ee9-406a-83c1-f16dbd6c0e84 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:04.533962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369662ed-2ee9-406a-83c1-f16dbd6c0e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:04.554043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-369662ed-2ee9-406a-83c1-f16dbd6c0e84 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:23:04.555295 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-369662ed-2ee9-406a-83c1-f16dbd6c0e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:04.589869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:23:04.624841 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 934/1852] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:23:04.641455 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26825bf-fbc3-42ca-9b08-d262948bd78f admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:04.642923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-f26825bf-fbc3-42ca-9b08-d262948bd78f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:04.649228 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:23:04.712345 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26825bf-fbc3-42ca-9b08-d262948bd78f admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:04.716016 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c081e-4075-4797-aca5-d5aab0fecd1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:04.716659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c081e-4075-4797-aca5-d5aab0fecd1c service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:04.716942 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c081e-4075-4797-aca5-d5aab0fecd1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:04.717218 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c081e-4075-4797-aca5-d5aab0fecd1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:04.731775 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c081e-4075-4797-aca5-d5aab0fecd1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:04.733345 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c081e-4075-4797-aca5-d5aab0fecd1c service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:04.744087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 919/1853] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:23:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:23:05.035451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369662ed-2ee9-406a-83c1-f16dbd6c0e84 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:05.037014 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-369662ed-2ee9-406a-83c1-f16dbd6c0e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:05.049500 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369662ed-2ee9-406a-83c1-f16dbd6c0e84 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:05.054596 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8cd53-89cd-4e3c-8dea-129fe5869c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:05.055298 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8cd53-89cd-4e3c-8dea-129fe5869c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:05.055796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8cd53-89cd-4e3c-8dea-129fe5869c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:05.056075 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8cd53-89cd-4e3c-8dea-129fe5869c2b admin admin] PATH_INFO: `/v3/projects/b2ab91f2df9943bc9c607d69660af3b2` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:05.057206 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8cd53-89cd-4e3c-8dea-129fe5869c2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2ab91f2df9943bc9c607d69660af3b2)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:05.059941 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8cd53-89cd-4e3c-8dea-129fe5869c2b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:05.109278 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-91b8cd53-89cd-4e3c-8dea-129fe5869c2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2ab91f2df9943bc9c607d69660af3b2', 'request_id': 'req-91b8cd53-89cd-4e3c-8dea-129fe5869c2b'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:05.115917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-91b8cd53-89cd-4e3c-8dea-129fe5869c2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2ab91f2df9943bc9c607d69660af3b2', 'request_id': 'req-91b8cd53-89cd-4e3c-8dea-129fe5869c2b'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:05.124173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-91b8cd53-89cd-4e3c-8dea-129fe5869c2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2ab91f2df9943bc9c607d69660af3b2', 'request_id': 'req-91b8cd53-89cd-4e3c-8dea-129fe5869c2b'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:05.134761 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-91b8cd53-89cd-4e3c-8dea-129fe5869c2b admin admin] The token cache is being invalidate because project b2ab91f2df9943bc9c607d69660af3b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:23:05.135045 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-91b8cd53-89cd-4e3c-8dea-129fe5869c2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:05.135685 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-91b8cd53-89cd-4e3c-8dea-129fe5869c2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:05.137160 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 935/1854] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:23:05 2022] DELETE /identity/v3/projects/b2ab91f2df9943bc9c607d69660af3b2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:23:05.142299 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cb223-a728-43f7-8582-704618270c96 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:05.142616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cb223-a728-43f7-8582-704618270c96 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:05.142896 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cb223-a728-43f7-8582-704618270c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:05.162842 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-542cb223-a728-43f7-8582-704618270c96 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:23:05.164094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-542cb223-a728-43f7-8582-704618270c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:05.223689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 920/1855] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:23:05.231718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a175b-0120-44b7-b73f-71af1ad0af40 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:05.232001 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a175b-0120-44b7-b73f-71af1ad0af40 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:05.232272 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a175b-0120-44b7-b73f-71af1ad0af40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:05.251983 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-c25a175b-0120-44b7-b73f-71af1ad0af40 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:23:05.253168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-c25a175b-0120-44b7-b73f-71af1ad0af40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:05.260859 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 936/1856] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:23:05.265108 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cb223-a728-43f7-8582-704618270c96 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:05.277942 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cb223-a728-43f7-8582-704618270c96 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:05.283126 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0d0eb-451f-46a1-94ae-ca6dcdd98d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:05.283761 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0d0eb-451f-46a1-94ae-ca6dcdd98d80 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:05.284041 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0d0eb-451f-46a1-94ae-ca6dcdd98d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:05.284310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0d0eb-451f-46a1-94ae-ca6dcdd98d80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:05.284951 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0d0eb-451f-46a1-94ae-ca6dcdd98d80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:23:05.285249 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0d0eb-451f-46a1-94ae-ca6dcdd98d80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:05.286815 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0d0eb-451f-46a1-94ae-ca6dcdd98d80 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:05.298796 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 921/1857] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:23:05.303088 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a175b-0120-44b7-b73f-71af1ad0af40 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:05.312298 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a175b-0120-44b7-b73f-71af1ad0af40 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:05.317190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deff860-9857-42d9-8425-15e1b16b5677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:05.317803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deff860-9857-42d9-8425-15e1b16b5677 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:05.318075 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deff860-9857-42d9-8425-15e1b16b5677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:05.318344 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deff860-9857-42d9-8425-15e1b16b5677 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:05.318983 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deff860-9857-42d9-8425-15e1b16b5677 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:05.320926 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deff860-9857-42d9-8425-15e1b16b5677 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:05.347741 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 937/1858] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:23:05 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:23:05.351916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0d0eb-451f-46a1-94ae-ca6dcdd98d80 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:05.363831 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0d0eb-451f-46a1-94ae-ca6dcdd98d80 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:05.424372 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce2ffb-2b98-4b01-8024-8bc2382bfb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:05.425020 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce2ffb-2b98-4b01-8024-8bc2382bfb7e admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:05.425235 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce2ffb-2b98-4b01-8024-8bc2382bfb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:05.425502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce2ffb-2b98-4b01-8024-8bc2382bfb7e admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:05.426131 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce2ffb-2b98-4b01-8024-8bc2382bfb7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:05.428063 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce2ffb-2b98-4b01-8024-8bc2382bfb7e admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:05.458497 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 922/1859] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:23:05 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:23:05.459915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deff860-9857-42d9-8425-15e1b16b5677 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:05.473853 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deff860-9857-42d9-8425-15e1b16b5677 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:05.478402 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c691a-c27e-42cf-8d4a-0eb774c7a3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:05.478993 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c691a-c27e-42cf-8d4a-0eb774c7a3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:05.479270 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c691a-c27e-42cf-8d4a-0eb774c7a3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:05.479584 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c691a-c27e-42cf-8d4a-0eb774c7a3a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:05.480201 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703c691a-c27e-42cf-8d4a-0eb774c7a3a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:05.485530 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703c691a-c27e-42cf-8d4a-0eb774c7a3a7 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:05.499134 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 938/1860] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:23:05.503125 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce2ffb-2b98-4b01-8024-8bc2382bfb7e admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:05.513153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce2ffb-2b98-4b01-8024-8bc2382bfb7e admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:05.518712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c08df-63e2-4748-a4fa-2c404eaa2b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:05.519307 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c08df-63e2-4748-a4fa-2c404eaa2b02 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:05.519613 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c08df-63e2-4748-a4fa-2c404eaa2b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:05.519889 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c08df-63e2-4748-a4fa-2c404eaa2b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:05.520520 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c08df-63e2-4748-a4fa-2c404eaa2b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:05.522100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c08df-63e2-4748-a4fa-2c404eaa2b02 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:05.534478 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 923/1861] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:23:05.538790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c691a-c27e-42cf-8d4a-0eb774c7a3a7 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:05.549232 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c691a-c27e-42cf-8d4a-0eb774c7a3a7 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:05.553785 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100517c-38ce-4f4b-9588-b74426a03edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:05.554453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100517c-38ce-4f4b-9588-b74426a03edb admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:05.554728 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100517c-38ce-4f4b-9588-b74426a03edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:05.554997 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100517c-38ce-4f4b-9588-b74426a03edb admin admin] PATH_INFO: `/v3/projects/434d61a7b1844a5180a564007c8b369f/users/07447907733f47a08c0a6554035058dc/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:05.569084 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8100517c-38ce-4f4b-9588-b74426a03edb admin admin] RBAC: Authorizing `identity:create_grant(project_id=434d61a7b1844a5180a564007c8b369f, user_id=07447907733f47a08c0a6554035058dc, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:05.571796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8100517c-38ce-4f4b-9588-b74426a03edb admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:05.585116 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 939/1862] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:23:05 2022] PUT /identity/v3/projects/434d61a7b1844a5180a564007c8b369f/users/07447907733f47a08c0a6554035058dc/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:23:05.590068 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6f108-acda-49d4-9571-af63a384f431 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:05.590372 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6f108-acda-49d4-9571-af63a384f431 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:05.590646 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6f108-acda-49d4-9571-af63a384f431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:05.609946 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-2ff6f108-acda-49d4-9571-af63a384f431 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:23:05.611198 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2ff6f108-acda-49d4-9571-af63a384f431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:05.616244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:23:05.670369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 924/1863] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:23:05.680706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100517c-38ce-4f4b-9588-b74426a03edb admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:05.682489 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8100517c-38ce-4f4b-9588-b74426a03edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:05.752165 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100517c-38ce-4f4b-9588-b74426a03edb admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:05.756969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3add3f-91e0-4be2-add6-9233ddf78a49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:05.757583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3add3f-91e0-4be2-add6-9233ddf78a49 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:05.757860 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3add3f-91e0-4be2-add6-9233ddf78a49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:05.758130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3add3f-91e0-4be2-add6-9233ddf78a49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:05.768400 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3add3f-91e0-4be2-add6-9233ddf78a49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:05.769876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3add3f-91e0-4be2-add6-9233ddf78a49 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:05.775167 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:23:05.780184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 940/1864] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:23:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:23:10.238429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f7f18-f47d-41ef-a76b-2a92e73cd46c None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:10.239373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f7f18-f47d-41ef-a76b-2a92e73cd46c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:10.239807 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f7f18-f47d-41ef-a76b-2a92e73cd46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:10.268571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-af4f7f18-f47d-41ef-a76b-2a92e73cd46c None None] MFA Rules not processed for user `07447907733f47a08c0a6554035058dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:23:10.269794 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-af4f7f18-f47d-41ef-a76b-2a92e73cd46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:10.339058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 925/1865] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:23:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:23:10.344016 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3add3f-91e0-4be2-add6-9233ddf78a49 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:10.354519 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3add3f-91e0-4be2-add6-9233ddf78a49 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:10.360890 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9fb5e-0809-4904-bacc-3faa686dce36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:10.361606 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9fb5e-0809-4904-bacc-3faa686dce36 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:10.362058 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9fb5e-0809-4904-bacc-3faa686dce36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:10.362335 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9fb5e-0809-4904-bacc-3faa686dce36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:10.363022 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9fb5e-0809-4904-bacc-3faa686dce36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:10.365676 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9fb5e-0809-4904-bacc-3faa686dce36 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:10.396301 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 941/1866] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:23:10 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:23:10.401741 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f7f18-f47d-41ef-a76b-2a92e73cd46c None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:10.414692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f7f18-f47d-41ef-a76b-2a92e73cd46c None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:10.479417 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adfd76-7f04-452e-a294-55a4a6aceb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:10.480214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adfd76-7f04-452e-a294-55a4a6aceb8b admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:10.480355 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adfd76-7f04-452e-a294-55a4a6aceb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:10.480630 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adfd76-7f04-452e-a294-55a4a6aceb8b admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:10.481284 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adfd76-7f04-452e-a294-55a4a6aceb8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:10.483281 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adfd76-7f04-452e-a294-55a4a6aceb8b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:10.511712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 926/1867] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:23:10 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:23:10.516276 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9fb5e-0809-4904-bacc-3faa686dce36 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:10.528658 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9fb5e-0809-4904-bacc-3faa686dce36 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:10.534467 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c405f2-f941-4136-9562-e7fee43ffed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:10.535223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c405f2-f941-4136-9562-e7fee43ffed3 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:10.535840 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c405f2-f941-4136-9562-e7fee43ffed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:10.535957 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c405f2-f941-4136-9562-e7fee43ffed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:10.536735 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c405f2-f941-4136-9562-e7fee43ffed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:10.538500 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c405f2-f941-4136-9562-e7fee43ffed3 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:10.555358 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 942/1868] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:23:10.557943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adfd76-7f04-452e-a294-55a4a6aceb8b admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:10.571075 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adfd76-7f04-452e-a294-55a4a6aceb8b admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:10.577296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b418543-f955-47cd-9ab6-51fc7844d41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:10.578322 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b418543-f955-47cd-9ab6-51fc7844d41b admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:10.578603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b418543-f955-47cd-9ab6-51fc7844d41b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:10.578874 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b418543-f955-47cd-9ab6-51fc7844d41b admin admin] PATH_INFO: `/v3/projects/7cdc3688c24547c6a97b874735746d5e/users/b948381f46104c1aa3ef0faceab9b444/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:10.598816 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b418543-f955-47cd-9ab6-51fc7844d41b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cdc3688c24547c6a97b874735746d5e, user_id=b948381f46104c1aa3ef0faceab9b444, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:10.601700 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b418543-f955-47cd-9ab6-51fc7844d41b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:10.617692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 927/1869] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:23:10 2022] PUT /identity/v3/projects/7cdc3688c24547c6a97b874735746d5e/users/b948381f46104c1aa3ef0faceab9b444/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:23:10.622262 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c405f2-f941-4136-9562-e7fee43ffed3 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:10.633804 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c405f2-f941-4136-9562-e7fee43ffed3 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:10.709705 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d9e72-b121-441d-9824-ad6fb113c7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:10.710357 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d9e72-b121-441d-9824-ad6fb113c7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:10.710641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d9e72-b121-441d-9824-ad6fb113c7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:10.710911 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d9e72-b121-441d-9824-ad6fb113c7e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:10.711723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d9e72-b121-441d-9824-ad6fb113c7e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:10.713353 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d9e72-b121-441d-9824-ad6fb113c7e4 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:10.728434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 943/1870] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:23:10.732086 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b418543-f955-47cd-9ab6-51fc7844d41b admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:10.745962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b418543-f955-47cd-9ab6-51fc7844d41b admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:10.750760 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f998a-adbb-4e4a-99c5-e82059054431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:10.751695 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f998a-adbb-4e4a-99c5-e82059054431 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:10.751988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f998a-adbb-4e4a-99c5-e82059054431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:10.752257 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f998a-adbb-4e4a-99c5-e82059054431 admin admin] PATH_INFO: `/v3/projects/7cdc3688c24547c6a97b874735746d5e/users/b948381f46104c1aa3ef0faceab9b444/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:10.754208 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f998a-adbb-4e4a-99c5-e82059054431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cdc3688c24547c6a97b874735746d5e, user_id=b948381f46104c1aa3ef0faceab9b444, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:10.757450 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f998a-adbb-4e4a-99c5-e82059054431 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:10.761351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:23:10.772039 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 928/1871] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:23:10 2022] PUT /identity/v3/projects/7cdc3688c24547c6a97b874735746d5e/users/b948381f46104c1aa3ef0faceab9b444/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:23:12.480634 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d9e72-b121-441d-9824-ad6fb113c7e4 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:12.483768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-bc8d9e72-b121-441d-9824-ad6fb113c7e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:12.498712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d9e72-b121-441d-9824-ad6fb113c7e4 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:12.616946 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce1270-fe91-4d24-bd80-f3e61f1a2e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:12.618979 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce1270-fe91-4d24-bd80-f3e61f1a2e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:12.619273 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce1270-fe91-4d24-bd80-f3e61f1a2e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:12.619736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce1270-fe91-4d24-bd80-f3e61f1a2e4f admin admin] PATH_INFO: `/v3/users/1132c97a102b408c8aaeca4ab9393bb4` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:12.621551 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce1270-fe91-4d24-bd80-f3e61f1a2e4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1132c97a102b408c8aaeca4ab9393bb4)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:12.623528 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce1270-fe91-4d24-bd80-f3e61f1a2e4f admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:12.705660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-59ce1270-fe91-4d24-bd80-f3e61f1a2e4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1132c97a102b408c8aaeca4ab9393bb4'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:12.722072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-59ce1270-fe91-4d24-bd80-f3e61f1a2e4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1132c97a102b408c8aaeca4ab9393bb4'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:12.728199 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:23:12.744899 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-59ce1270-fe91-4d24-bd80-f3e61f1a2e4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1132c97a102b408c8aaeca4ab9393bb4'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:12.746011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-59ce1270-fe91-4d24-bd80-f3e61f1a2e4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1132c97a102b408c8aaeca4ab9393bb4'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:12.755056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-59ce1270-fe91-4d24-bd80-f3e61f1a2e4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1132c97a102b408c8aaeca4ab9393bb4'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:12.762779 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 944/1872] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:23:12 2022] DELETE /identity/v3/users/1132c97a102b408c8aaeca4ab9393bb4 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:23:12.767323 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626af90-63e5-4b75-9de7-fb1499393eb4 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:12.771018 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626af90-63e5-4b75-9de7-fb1499393eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:12.771640 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626af90-63e5-4b75-9de7-fb1499393eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:12.802965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-b626af90-63e5-4b75-9de7-fb1499393eb4 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:23:12.804392 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-b626af90-63e5-4b75-9de7-fb1499393eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:12.906725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 929/1873] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:23:12.916754 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce1270-fe91-4d24-bd80-f3e61f1a2e4f admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:12.918342 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-59ce1270-fe91-4d24-bd80-f3e61f1a2e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:12.999348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce1270-fe91-4d24-bd80-f3e61f1a2e4f admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:13.004811 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2e089-b362-437a-96d4-5a555a5cd0e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:13.007147 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2e089-b362-437a-96d4-5a555a5cd0e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:13.007147 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2e089-b362-437a-96d4-5a555a5cd0e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:13.007147 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2e089-b362-437a-96d4-5a555a5cd0e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:13.017976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2e089-b362-437a-96d4-5a555a5cd0e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:13.019853 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2e089-b362-437a-96d4-5a555a5cd0e2 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:13.031008 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 945/1874] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:23:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:23:13.759417 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626af90-63e5-4b75-9de7-fb1499393eb4 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:13.761284 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-b626af90-63e5-4b75-9de7-fb1499393eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:13.786291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626af90-63e5-4b75-9de7-fb1499393eb4 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:13.792822 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e803e5-2bcc-4f6b-91d0-809388ac646b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:13.793817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e803e5-2bcc-4f6b-91d0-809388ac646b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:13.794303 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e803e5-2bcc-4f6b-91d0-809388ac646b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:13.794851 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e803e5-2bcc-4f6b-91d0-809388ac646b admin admin] PATH_INFO: `/v3/projects/75517fe5629c4f388aca45e08e49ae89` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:13.796280 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e803e5-2bcc-4f6b-91d0-809388ac646b admin admin] RBAC: Authorizing `identity:delete_project(project_id=75517fe5629c4f388aca45e08e49ae89)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:13.798745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e803e5-2bcc-4f6b-91d0-809388ac646b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:13.869743 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-26e803e5-2bcc-4f6b-91d0-809388ac646b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75517fe5629c4f388aca45e08e49ae89', 'request_id': 'req-26e803e5-2bcc-4f6b-91d0-809388ac646b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:13.884671 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-26e803e5-2bcc-4f6b-91d0-809388ac646b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75517fe5629c4f388aca45e08e49ae89', 'request_id': 'req-26e803e5-2bcc-4f6b-91d0-809388ac646b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:13.893994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-26e803e5-2bcc-4f6b-91d0-809388ac646b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75517fe5629c4f388aca45e08e49ae89', 'request_id': 'req-26e803e5-2bcc-4f6b-91d0-809388ac646b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:13.910528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-26e803e5-2bcc-4f6b-91d0-809388ac646b admin admin] The token cache is being invalidate because project 75517fe5629c4f388aca45e08e49ae89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:23:13.910879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-26e803e5-2bcc-4f6b-91d0-809388ac646b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:13.911483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-26e803e5-2bcc-4f6b-91d0-809388ac646b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:23:13.916845 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 930/1875] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:23:13 2022] DELETE /identity/v3/projects/75517fe5629c4f388aca45e08e49ae89 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:23:13.918261 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f364bf9-d8b5-4677-85f4-b55fd3b0e26b None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:13.918744 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f364bf9-d8b5-4677-85f4-b55fd3b0e26b None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:13.918822 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f364bf9-d8b5-4677-85f4-b55fd3b0e26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:13.945426 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-1f364bf9-d8b5-4677-85f4-b55fd3b0e26b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:23:13.947500 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-1f364bf9-d8b5-4677-85f4-b55fd3b0e26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:14.029447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 946/1876] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:23:14.037194 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03bb54-e779-405c-89d3-cdabbd840b37 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:14.037474 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03bb54-e779-405c-89d3-cdabbd840b37 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:14.038048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03bb54-e779-405c-89d3-cdabbd840b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:14.078074 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-5d03bb54-e779-405c-89d3-cdabbd840b37 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:23:14.084566 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-5d03bb54-e779-405c-89d3-cdabbd840b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:14.100584 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 931/1877] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:23:14.101510 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f364bf9-d8b5-4677-85f4-b55fd3b0e26b None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:14.115322 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f364bf9-d8b5-4677-85f4-b55fd3b0e26b None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:14.133056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e1d02-e38a-4ed7-85f5-84e2aee53433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:14.135039 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e1d02-e38a-4ed7-85f5-84e2aee53433 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:14.135039 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e1d02-e38a-4ed7-85f5-84e2aee53433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:14.135039 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e1d02-e38a-4ed7-85f5-84e2aee53433 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:14.135700 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e1d02-e38a-4ed7-85f5-84e2aee53433 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:23:14.136727 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e1d02-e38a-4ed7-85f5-84e2aee53433 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:14.138765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e1d02-e38a-4ed7-85f5-84e2aee53433 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:14.181236 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 947/1878] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:23:14.185730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03bb54-e779-405c-89d3-cdabbd840b37 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:14.200961 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03bb54-e779-405c-89d3-cdabbd840b37 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:14.209058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e50fe1-d4c4-4fd9-9b64-e1974f9b6f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:14.210506 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e50fe1-d4c4-4fd9-9b64-e1974f9b6f88 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:14.210952 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e50fe1-d4c4-4fd9-9b64-e1974f9b6f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:14.211540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e50fe1-d4c4-4fd9-9b64-e1974f9b6f88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:14.212469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e50fe1-d4c4-4fd9-9b64-e1974f9b6f88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:14.214993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e50fe1-d4c4-4fd9-9b64-e1974f9b6f88 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:14.278022 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e1d02-e38a-4ed7-85f5-84e2aee53433 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:14.286645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 932/1879] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:23:14 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:23:14.296777 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e1d02-e38a-4ed7-85f5-84e2aee53433 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:14.387547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2afd2-5b3e-4dd7-8472-7f9c61c18c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:14.388686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2afd2-5b3e-4dd7-8472-7f9c61c18c15 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:14.389199 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2afd2-5b3e-4dd7-8472-7f9c61c18c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:14.389496 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2afd2-5b3e-4dd7-8472-7f9c61c18c15 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:14.390521 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2afd2-5b3e-4dd7-8472-7f9c61c18c15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:14.393976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2afd2-5b3e-4dd7-8472-7f9c61c18c15 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:14.432726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 948/1880] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:23:14 2022] POST /identity/v3/users => generated 292 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:23:14.434455 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e50fe1-d4c4-4fd9-9b64-e1974f9b6f88 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:14.452529 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e50fe1-d4c4-4fd9-9b64-e1974f9b6f88 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:14.460181 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e81048-5152-4aaa-bdb9-69c1e48bcbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:14.460960 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e81048-5152-4aaa-bdb9-69c1e48bcbab admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:14.461337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e81048-5152-4aaa-bdb9-69c1e48bcbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:14.461728 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e81048-5152-4aaa-bdb9-69c1e48bcbab admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:14.463223 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e81048-5152-4aaa-bdb9-69c1e48bcbab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:14.465161 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e81048-5152-4aaa-bdb9-69c1e48bcbab admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:14.481059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2afd2-5b3e-4dd7-8472-7f9c61c18c15 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:14.488517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:23:14.492920 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 933/1881] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:23:14.500590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2afd2-5b3e-4dd7-8472-7f9c61c18c15 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:14.511877 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed4f9e-928f-41d4-abab-ce58be1569d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:14.513246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed4f9e-928f-41d4-abab-ce58be1569d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:14.513977 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed4f9e-928f-41d4-abab-ce58be1569d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:14.514614 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed4f9e-928f-41d4-abab-ce58be1569d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:14.515495 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed4f9e-928f-41d4-abab-ce58be1569d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:14.518081 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed4f9e-928f-41d4-abab-ce58be1569d8 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:14.534986 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 949/1882] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:23:14.541162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e81048-5152-4aaa-bdb9-69c1e48bcbab admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:14.554103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e81048-5152-4aaa-bdb9-69c1e48bcbab admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:14.560512 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43bcbb-9486-4314-93be-e6895309038a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:14.561611 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43bcbb-9486-4314-93be-e6895309038a admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:14.562245 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43bcbb-9486-4314-93be-e6895309038a admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:14.562723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43bcbb-9486-4314-93be-e6895309038a admin admin] PATH_INFO: `/v3/projects/dcfe635652d548098c058c5fbf1319c2/users/8ecc2579ccf9406383a6eb3a0b9356cb/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:14.602642 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43bcbb-9486-4314-93be-e6895309038a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcfe635652d548098c058c5fbf1319c2, user_id=8ecc2579ccf9406383a6eb3a0b9356cb, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:14.607515 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43bcbb-9486-4314-93be-e6895309038a admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:14.640935 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 934/1883] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:23:14 2022] PUT /identity/v3/projects/dcfe635652d548098c058c5fbf1319c2/users/8ecc2579ccf9406383a6eb3a0b9356cb/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:23:14.646222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296be75-e83a-4213-b7e3-e2786e789779 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:14.646712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296be75-e83a-4213-b7e3-e2786e789779 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:14.646993 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296be75-e83a-4213-b7e3-e2786e789779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:14.678168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-2296be75-e83a-4213-b7e3-e2786e789779 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:23:14.681101 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2296be75-e83a-4213-b7e3-e2786e789779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:14.789591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 950/1884] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:23:14.800680 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43bcbb-9486-4314-93be-e6895309038a admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:14.804515 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8d43bcbb-9486-4314-93be-e6895309038a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:14.827382 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:23:14.900596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43bcbb-9486-4314-93be-e6895309038a admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:14.913715 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc632dac-97ca-44d2-ac33-4d19754e6923 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:14.914689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc632dac-97ca-44d2-ac33-4d19754e6923 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:14.915769 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc632dac-97ca-44d2-ac33-4d19754e6923 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:14.916267 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc632dac-97ca-44d2-ac33-4d19754e6923 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:14.929701 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc632dac-97ca-44d2-ac33-4d19754e6923 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:14.931842 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc632dac-97ca-44d2-ac33-4d19754e6923 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:14.945681 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 935/1885] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:23:16.402756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf53e6c-b2b4-40e9-8b9a-95f2e7112e7a None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:16.403339 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf53e6c-b2b4-40e9-8b9a-95f2e7112e7a None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:16.403339 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf53e6c-b2b4-40e9-8b9a-95f2e7112e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:16.435795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-8bf53e6c-b2b4-40e9-8b9a-95f2e7112e7a None None] MFA Rules not processed for user `8ecc2579ccf9406383a6eb3a0b9356cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:23:16.437386 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8bf53e6c-b2b4-40e9-8b9a-95f2e7112e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:16.518472 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 951/1886] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:23:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:23:16.532011 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc632dac-97ca-44d2-ac33-4d19754e6923 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:16.533573 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-fc632dac-97ca-44d2-ac33-4d19754e6923 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:16.601749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc632dac-97ca-44d2-ac33-4d19754e6923 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:16.605978 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f39b1c-3c8a-4032-9453-3552522d0460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:16.606456 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f39b1c-3c8a-4032-9453-3552522d0460 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:16.606938 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f39b1c-3c8a-4032-9453-3552522d0460 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:16.607281 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f39b1c-3c8a-4032-9453-3552522d0460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:16.628744 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f39b1c-3c8a-4032-9453-3552522d0460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:16.630197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f39b1c-3c8a-4032-9453-3552522d0460 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:16.642280 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 936/1887] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:23:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:23:16.949135 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647adf8c-9d91-4fbf-842d-447936811c82 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:16.949596 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647adf8c-9d91-4fbf-842d-447936811c82 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:16.950049 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647adf8c-9d91-4fbf-842d-447936811c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:16.988095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-647adf8c-9d91-4fbf-842d-447936811c82 None None] MFA Rules not processed for user `b948381f46104c1aa3ef0faceab9b444`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:23:16.989624 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-647adf8c-9d91-4fbf-842d-447936811c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:17.169517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 952/1888] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:23:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:23:17.187673 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f39b1c-3c8a-4032-9453-3552522d0460 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:23:17.189416 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-71f39b1c-3c8a-4032-9453-3552522d0460 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:17.318305 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f39b1c-3c8a-4032-9453-3552522d0460 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:23:17.324400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734cc26-df48-49c2-a3ee-12005c5f27ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:23:17.325039 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734cc26-df48-49c2-a3ee-12005c5f27ef service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:17.325314 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734cc26-df48-49c2-a3ee-12005c5f27ef service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:17.325585 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734cc26-df48-49c2-a3ee-12005c5f27ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:17.327331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-b734cc26-df48-49c2-a3ee-12005c5f27ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:17.407723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:23:17.412155 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734cc26-df48-49c2-a3ee-12005c5f27ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:23:17.413845 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734cc26-df48-49c2-a3ee-12005c5f27ef service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:23:17.424833 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 937/1889] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:23:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:23:18.218974 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6004023-48e8-4ec7-9ba5-7117b1ed061e None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:18.219498 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6004023-48e8-4ec7-9ba5-7117b1ed061e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:18.219642 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6004023-48e8-4ec7-9ba5-7117b1ed061e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:18.234499 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:23:18.248104 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-b6004023-48e8-4ec7-9ba5-7117b1ed061e None None] MFA Rules not processed for user `8ecc2579ccf9406383a6eb3a0b9356cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:23:18.249995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b6004023-48e8-4ec7-9ba5-7117b1ed061e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:18.267321 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 953/1890] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:23:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:23:19.352517 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec6f3f-2f79-4c57-9b67-3900dccf3cb6 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:19.353578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec6f3f-2f79-4c57-9b67-3900dccf3cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:19.354124 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec6f3f-2f79-4c57-9b67-3900dccf3cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:19.390693 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-d6ec6f3f-2f79-4c57-9b67-3900dccf3cb6 None None] MFA Rules not processed for user `e550c842cdbd4da396045c50a6d38866`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:23:19.393615 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d6ec6f3f-2f79-4c57-9b67-3900dccf3cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:19.471161 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 938/1891] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:23:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:23:29.463216 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=27 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:23:45.533533 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32573a7-cc0b-4246-bf5c-06520537e542 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:23:45.534065 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32573a7-cc0b-4246-bf5c-06520537e542 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:23:45.534065 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32573a7-cc0b-4246-bf5c-06520537e542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:23:45.553080 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-b32573a7-cc0b-4246-bf5c-06520537e542 None None] MFA Rules not processed for user `07447907733f47a08c0a6554035058dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:23:45.554367 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b32573a7-cc0b-4246-bf5c-06520537e542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:23:45.561290 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 954/1892] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:23:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:23:55.547905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:24:06.729607 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec6f3f-2f79-4c57-9b67-3900dccf3cb6 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:24:06.734666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d6ec6f3f-2f79-4c57-9b67-3900dccf3cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:24:06.749045 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec6f3f-2f79-4c57-9b67-3900dccf3cb6 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:24:06.752978 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9ca41-f05f-4e57-9cfc-76f62dfe183d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:24:06.753756 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9ca41-f05f-4e57-9cfc-76f62dfe183d service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:24:06.754040 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9ca41-f05f-4e57-9cfc-76f62dfe183d service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:24:06.754312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9ca41-f05f-4e57-9cfc-76f62dfe183d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:24:06.763407 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9ca41-f05f-4e57-9cfc-76f62dfe183d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:24:06.764853 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9ca41-f05f-4e57-9cfc-76f62dfe183d service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:24:06.774394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 939/1893] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:24:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:24:16.769953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=6 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:25:12.829918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32573a7-cc0b-4246-bf5c-06520537e542 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:25:12.831806 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b32573a7-cc0b-4246-bf5c-06520537e542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:25:12.916826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32573a7-cc0b-4246-bf5c-06520537e542 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:25:12.924609 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277209cb-a433-422c-86de-4cfd90796d83 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:25:12.925227 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277209cb-a433-422c-86de-4cfd90796d83 service placement] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:25:12.925507 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277209cb-a433-422c-86de-4cfd90796d83 service placement] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:25:12.925777 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277209cb-a433-422c-86de-4cfd90796d83 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:25:12.928029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-277209cb-a433-422c-86de-4cfd90796d83 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:25:12.942458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277209cb-a433-422c-86de-4cfd90796d83 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:25:12.944270 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277209cb-a433-422c-86de-4cfd90796d83 service placement] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:25:12.963828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 955/1894] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:25:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:25:14.062122 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0649b0-43f1-4870-92e0-3621935832b9 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:25:14.062629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0649b0-43f1-4870-92e0-3621935832b9 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:25:14.062732 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0649b0-43f1-4870-92e0-3621935832b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:25:14.081709 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-0a0649b0-43f1-4870-92e0-3621935832b9 None None] MFA Rules not processed for user `00374760bfa648e48dedae8aadb61c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:25:14.082823 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-0a0649b0-43f1-4870-92e0-3621935832b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:25:14.135256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 940/1895] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:25:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:25:21.458110 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448f225-dd14-435f-b230-f1566da238c6 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:25:21.458711 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448f225-dd14-435f-b230-f1566da238c6 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:25:21.458711 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1448f225-dd14-435f-b230-f1566da238c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:25:21.480254 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-1448f225-dd14-435f-b230-f1566da238c6 None None] MFA Rules not processed for user `2346003afe74477496045e9bc72173d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:25:21.481545 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-1448f225-dd14-435f-b230-f1566da238c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:25:21.549889 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 956/1896] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Oct 21 21:25:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:25:21.602286 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0649b0-43f1-4870-92e0-3621935832b9 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:25:21.603757 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-0a0649b0-43f1-4870-92e0-3621935832b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:25:21.679366 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0649b0-43f1-4870-92e0-3621935832b9 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:25:21.684394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db761a-95fb-4bcb-a196-90d5b92d1065 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ce720c8df014f9a8af062138aecd1b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:25:21.685577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db761a-95fb-4bcb-a196-90d5b92d1065 service swift] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:25:21.685854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db761a-95fb-4bcb-a196-90d5b92d1065 service swift] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:25:21.686124 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db761a-95fb-4bcb-a196-90d5b92d1065 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:25:21.694351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db761a-95fb-4bcb-a196-90d5b92d1065 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:25:21.697038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db761a-95fb-4bcb-a196-90d5b92d1065 service swift] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:25:21.705416 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 941/1897] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Oct 21 21:25:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 21:25:31.543990 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=57 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:25:31.702085 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=57 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:25:37.224536 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56675367-37d6-40b3-ad0c-8fc1be4313be None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:25:37.225103 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56675367-37d6-40b3-ad0c-8fc1be4313be None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:25:37.225203 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56675367-37d6-40b3-ad0c-8fc1be4313be None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:25:37.226725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 957/1898] 10.222.0.13 () {54 vars in 921 bytes} [Fri Oct 21 21:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:25:37.233187 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d6c8f-ffa5-472e-8778-66e13ecfca00 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:25:37.233707 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d6c8f-ffa5-472e-8778-66e13ecfca00 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:25:37.233807 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d6c8f-ffa5-472e-8778-66e13ecfca00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:25:37.256327 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-177d6c8f-ffa5-472e-8778-66e13ecfca00 None None] MFA Rules not processed for user `12dad9f016a740b98538722e1b723823`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:25:37.257588 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-177d6c8f-ffa5-472e-8778-66e13ecfca00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:25:37.267467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 942/1899] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Oct 21 21:25:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:25:37.279774 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56675367-37d6-40b3-ad0c-8fc1be4313be None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:25:37.281440 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-56675367-37d6-40b3-ad0c-8fc1be4313be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:25:37.338808 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56675367-37d6-40b3-ad0c-8fc1be4313be None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:25:37.343010 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3325823-a648-4ef5-ba61-3dd3433914b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce5ec441b06408bbadd6fc040af0da5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:25:37.343869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3325823-a648-4ef5-ba61-3dd3433914b8 service glance] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:25:37.344160 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3325823-a648-4ef5-ba61-3dd3433914b8 service glance] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:25:37.344495 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3325823-a648-4ef5-ba61-3dd3433914b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:25:37.354971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3325823-a648-4ef5-ba61-3dd3433914b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:25:37.356473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3325823-a648-4ef5-ba61-3dd3433914b8 service glance] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:25:37.367273 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 958/1900] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:25:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:25:37.435402 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b8c68-2c64-4e59-9460-0315a76518b3 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:25:37.435639 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b8c68-2c64-4e59-9460-0315a76518b3 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:25:37.435935 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b8c68-2c64-4e59-9460-0315a76518b3 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:25:37.437303 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 943/1901] 10.222.0.13 () {54 vars in 921 bytes} [Fri Oct 21 21:25:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:25:37.442115 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee6bf2-6700-4497-9c71-45a508aa24d8 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:25:37.442415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee6bf2-6700-4497-9c71-45a508aa24d8 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:25:37.442688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee6bf2-6700-4497-9c71-45a508aa24d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:25:37.462325 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-49ee6bf2-6700-4497-9c71-45a508aa24d8 None None] MFA Rules not processed for user `d34dc4e7c5474b6b96977019328d8890`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:25:37.463519 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-49ee6bf2-6700-4497-9c71-45a508aa24d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:25:37.473943 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 959/1902] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Oct 21 21:25:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:25:37.486293 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b8c68-2c64-4e59-9460-0315a76518b3 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:25:37.497110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b8c68-2c64-4e59-9460-0315a76518b3 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:25:37.500571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953306d7-d9ac-45ac-8da9-2eaa2d050521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:25:37.501159 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953306d7-d9ac-45ac-8da9-2eaa2d050521 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:25:37.501430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953306d7-d9ac-45ac-8da9-2eaa2d050521 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:25:37.501695 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953306d7-d9ac-45ac-8da9-2eaa2d050521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:25:37.508551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953306d7-d9ac-45ac-8da9-2eaa2d050521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:25:37.509948 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953306d7-d9ac-45ac-8da9-2eaa2d050521 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:25:37.517567 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 944/1903] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:25:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:25:37.598891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b287e-918d-4d78-a0d2-cefab891302f None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:25:37.599173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b287e-918d-4d78-a0d2-cefab891302f None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:25:37.599443 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b287e-918d-4d78-a0d2-cefab891302f None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:25:37.600889 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 960/1904] 10.222.0.13 () {54 vars in 921 bytes} [Fri Oct 21 21:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:25:37.606670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bcbee-e011-4709-8077-2f99fa9e7845 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:25:37.606948 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bcbee-e011-4709-8077-2f99fa9e7845 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:25:37.607213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bcbee-e011-4709-8077-2f99fa9e7845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:25:37.626622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-d22bcbee-e011-4709-8077-2f99fa9e7845 None None] MFA Rules not processed for user `12dad9f016a740b98538722e1b723823`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:25:37.627852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d22bcbee-e011-4709-8077-2f99fa9e7845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:25:37.637162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 945/1905] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Oct 21 21:25:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:25:37.647390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b287e-918d-4d78-a0d2-cefab891302f None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:25:37.660997 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b287e-918d-4d78-a0d2-cefab891302f None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:25:37.664605 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958bb64-5ce8-41c9-99fc-fa91a3ceae8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:25:37.665213 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958bb64-5ce8-41c9-99fc-fa91a3ceae8f service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:25:37.665493 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958bb64-5ce8-41c9-99fc-fa91a3ceae8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:25:37.665763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958bb64-5ce8-41c9-99fc-fa91a3ceae8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:25:37.676749 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958bb64-5ce8-41c9-99fc-fa91a3ceae8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:25:37.678173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958bb64-5ce8-41c9-99fc-fa91a3ceae8f service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:25:37.689184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 961/1906] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:25:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:25:44.309094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bcbee-e011-4709-8077-2f99fa9e7845 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:25:44.318440 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bcbee-e011-4709-8077-2f99fa9e7845 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:25:44.322292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c1d64-7b37-445b-93c5-ba15c0bf3018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:25:44.322899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c1d64-7b37-445b-93c5-ba15c0bf3018 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:25:44.323182 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c1d64-7b37-445b-93c5-ba15c0bf3018 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:25:44.323456 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c1d64-7b37-445b-93c5-ba15c0bf3018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:25:44.325268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9b0c1d64-7b37-445b-93c5-ba15c0bf3018 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:25:44.338287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c1d64-7b37-445b-93c5-ba15c0bf3018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:25:44.339718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c1d64-7b37-445b-93c5-ba15c0bf3018 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:25:44.350904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 946/1907] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:25:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:25:45.550172 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958bb64-5ce8-41c9-99fc-fa91a3ceae8f service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:25:45.563918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958bb64-5ce8-41c9-99fc-fa91a3ceae8f service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:25:45.567471 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a2971-8ced-422d-9b19-a8571bda9dfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:25:45.568101 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a2971-8ced-422d-9b19-a8571bda9dfb service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:25:45.568378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a2971-8ced-422d-9b19-a8571bda9dfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:25:45.568663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a2971-8ced-422d-9b19-a8571bda9dfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:25:45.570371 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e86a2971-8ced-422d-9b19-a8571bda9dfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:25:45.583905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a2971-8ced-422d-9b19-a8571bda9dfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:25:45.585361 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a2971-8ced-422d-9b19-a8571bda9dfb service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:25:45.596592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 962/1908] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:25:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:25:54.346090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=18 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:25:55.591880 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=41 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:25:56.408464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c1d64-7b37-445b-93c5-ba15c0bf3018 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:25:56.419531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c1d64-7b37-445b-93c5-ba15c0bf3018 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:25:56.425096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a481850-5141-423e-980e-06a564bdd0dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:25:56.425717 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a481850-5141-423e-980e-06a564bdd0dd service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:25:56.425998 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a481850-5141-423e-980e-06a564bdd0dd service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:25:56.426270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a481850-5141-423e-980e-06a564bdd0dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:25:56.427994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1a481850-5141-423e-980e-06a564bdd0dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:25:56.441159 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a481850-5141-423e-980e-06a564bdd0dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:25:56.442718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a481850-5141-423e-980e-06a564bdd0dd service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:25:56.452373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 947/1909] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:25:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:26:03.218004 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a2971-8ced-422d-9b19-a8571bda9dfb service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:26:03.229036 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a2971-8ced-422d-9b19-a8571bda9dfb service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:26:03.234539 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc62ea-90ac-413b-9fc4-286b7896e95b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:26:03.235142 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc62ea-90ac-413b-9fc4-286b7896e95b service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:26:03.235420 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc62ea-90ac-413b-9fc4-286b7896e95b service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:26:03.235803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc62ea-90ac-413b-9fc4-286b7896e95b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:26:03.237508 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-49bc62ea-90ac-413b-9fc4-286b7896e95b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:26:03.294924 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc62ea-90ac-413b-9fc4-286b7896e95b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:26:03.296535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc62ea-90ac-413b-9fc4-286b7896e95b service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:26:03.305423 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 963/1910] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:26:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:26:05.592750 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a481850-5141-423e-980e-06a564bdd0dd service nova] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:26:05.594149 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1a481850-5141-423e-980e-06a564bdd0dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:26:05.606022 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a481850-5141-423e-980e-06a564bdd0dd service nova] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:26:05.609530 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafa531-dae5-4a27-a1b2-c071b3e873d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:26:05.610116 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafa531-dae5-4a27-a1b2-c071b3e873d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:26:05.610388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafa531-dae5-4a27-a1b2-c071b3e873d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:26:05.610653 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafa531-dae5-4a27-a1b2-c071b3e873d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:26:05.612302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ebafa531-dae5-4a27-a1b2-c071b3e873d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:26:05.671846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebafa531-dae5-4a27-a1b2-c071b3e873d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:26:05.673305 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebafa531-dae5-4a27-a1b2-c071b3e873d8 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:26:05.682485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 948/1911] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:26:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:26:13.301404 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=28 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:26:15.678103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=34 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:26:59.457748 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc62ea-90ac-413b-9fc4-286b7896e95b service nova] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:26:59.469459 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc62ea-90ac-413b-9fc4-286b7896e95b service nova] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:26:59.474799 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955eda04-59b9-4d6f-8e77-e460a03c43c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:26:59.475391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955eda04-59b9-4d6f-8e77-e460a03c43c9 service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:26:59.475698 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955eda04-59b9-4d6f-8e77-e460a03c43c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:26:59.475971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955eda04-59b9-4d6f-8e77-e460a03c43c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:26:59.477713 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-955eda04-59b9-4d6f-8e77-e460a03c43c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:26:59.488311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955eda04-59b9-4d6f-8e77-e460a03c43c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:26:59.489883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955eda04-59b9-4d6f-8e77-e460a03c43c9 service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:26:59.498559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 964/1912] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:26:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:27:01.045240 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafa531-dae5-4a27-a1b2-c071b3e873d8 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:27:01.054805 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafa531-dae5-4a27-a1b2-c071b3e873d8 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:27:01.059092 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382f519-1d71-4bab-83ca-393bf33ac052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:27:01.059732 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382f519-1d71-4bab-83ca-393bf33ac052 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:27:01.060011 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382f519-1d71-4bab-83ca-393bf33ac052 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:27:01.060282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382f519-1d71-4bab-83ca-393bf33ac052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:27:01.062023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d382f519-1d71-4bab-83ca-393bf33ac052 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:27:01.071691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382f519-1d71-4bab-83ca-393bf33ac052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:27:01.073652 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382f519-1d71-4bab-83ca-393bf33ac052 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:27:01.081854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 949/1913] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:27:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:27:03.066021 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955eda04-59b9-4d6f-8e77-e460a03c43c9 service nova] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:27:03.078720 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955eda04-59b9-4d6f-8e77-e460a03c43c9 service nova] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:27:03.086081 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd24c2c-0ac1-4e45-88d7-9bc0da4cc94d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:27:03.086711 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd24c2c-0ac1-4e45-88d7-9bc0da4cc94d service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:27:03.087074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd24c2c-0ac1-4e45-88d7-9bc0da4cc94d service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:27:03.087936 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd24c2c-0ac1-4e45-88d7-9bc0da4cc94d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:27:03.095584 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ddd24c2c-0ac1-4e45-88d7-9bc0da4cc94d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:27:03.109093 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd24c2c-0ac1-4e45-88d7-9bc0da4cc94d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:27:03.110811 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd24c2c-0ac1-4e45-88d7-9bc0da4cc94d service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:27:03.133124 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 965/1914] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:27:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:27:11.078194 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=6 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:27:13.117394 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=12 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:28:17.368825 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382f519-1d71-4bab-83ca-393bf33ac052 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:28:17.382317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382f519-1d71-4bab-83ca-393bf33ac052 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:28:17.387719 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6e05a-58ea-4956-a472-cd53dff54c32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:28:17.388316 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6e05a-58ea-4956-a472-cd53dff54c32 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:28:17.388591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6e05a-58ea-4956-a472-cd53dff54c32 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:28:17.388857 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6e05a-58ea-4956-a472-cd53dff54c32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:28:17.396344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6e05a-58ea-4956-a472-cd53dff54c32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:28:17.398154 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6e05a-58ea-4956-a472-cd53dff54c32 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:28:17.406736 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 950/1915] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:28:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:28:27.402788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=6 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:29:08.463349 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd24c2c-0ac1-4e45-88d7-9bc0da4cc94d service nova] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:29:08.481597 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd24c2c-0ac1-4e45-88d7-9bc0da4cc94d service nova] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:29:08.485142 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e0e41-b93b-4d27-8ab5-3793bcc556a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:29:08.485738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e0e41-b93b-4d27-8ab5-3793bcc556a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:29:08.486015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e0e41-b93b-4d27-8ab5-3793bcc556a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:29:08.486286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e0e41-b93b-4d27-8ab5-3793bcc556a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:29:08.509063 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e0e41-b93b-4d27-8ab5-3793bcc556a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:29:08.510534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e0e41-b93b-4d27-8ab5-3793bcc556a7 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:29:08.520021 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 966/1916] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:29:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:29:18.515891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=17 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:30:45.206655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6e05a-58ea-4956-a472-cd53dff54c32 service nova] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:30:45.245990 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6e05a-58ea-4956-a472-cd53dff54c32 service nova] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:30:45.249769 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1d20c-8d83-4384-9847-ef50adf7634f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:30:45.250451 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1d20c-8d83-4384-9847-ef50adf7634f service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:30:45.250728 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1d20c-8d83-4384-9847-ef50adf7634f service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:30:45.250997 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1d20c-8d83-4384-9847-ef50adf7634f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:30:45.281069 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1d20c-8d83-4384-9847-ef50adf7634f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:30:45.283173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1d20c-8d83-4384-9847-ef50adf7634f service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:30:45.307018 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 951/1917] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:30:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:30:46.495302 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e0e41-b93b-4d27-8ab5-3793bcc556a7 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:30:46.505265 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e0e41-b93b-4d27-8ab5-3793bcc556a7 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:30:46.508986 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24862f5-1994-414a-8c2b-043c6117a7e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:30:46.509587 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24862f5-1994-414a-8c2b-043c6117a7e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:30:46.509860 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24862f5-1994-414a-8c2b-043c6117a7e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:30:46.510126 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24862f5-1994-414a-8c2b-043c6117a7e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:30:46.520825 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24862f5-1994-414a-8c2b-043c6117a7e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:30:46.522478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24862f5-1994-414a-8c2b-043c6117a7e4 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:30:46.536138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 967/1918] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:30:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:30:55.304115 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=28 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:30:56.529226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=6 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:30:59.142856 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1d20c-8d83-4384-9847-ef50adf7634f service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:30:59.162907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1d20c-8d83-4384-9847-ef50adf7634f service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:30:59.167990 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7015e3-a7ad-43b3-879d-a3e3574cf242 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:30:59.168600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7015e3-a7ad-43b3-879d-a3e3574cf242 service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:30:59.168872 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7015e3-a7ad-43b3-879d-a3e3574cf242 service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:30:59.169136 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7015e3-a7ad-43b3-879d-a3e3574cf242 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:30:59.177592 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7015e3-a7ad-43b3-879d-a3e3574cf242 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:30:59.179174 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7015e3-a7ad-43b3-879d-a3e3574cf242 service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:30:59.189275 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 952/1919] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:30:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:31:02.956929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24862f5-1994-414a-8c2b-043c6117a7e4 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:31:02.958313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f24862f5-1994-414a-8c2b-043c6117a7e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:31:02.994320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24862f5-1994-414a-8c2b-043c6117a7e4 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:31:03.000262 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ca635-e5a7-4ca2-8e1b-678ba0c49773 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:31:03.000856 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ca635-e5a7-4ca2-8e1b-678ba0c49773 service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:31:03.001134 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ca635-e5a7-4ca2-8e1b-678ba0c49773 service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:31:03.001402 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ca635-e5a7-4ca2-8e1b-678ba0c49773 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:31:03.008937 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524ca635-e5a7-4ca2-8e1b-678ba0c49773 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:31:03.010496 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524ca635-e5a7-4ca2-8e1b-678ba0c49773 service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:31:03.037385 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 968/1920] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:31:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:31:08.668162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7015e3-a7ad-43b3-879d-a3e3574cf242 service placement] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:31:08.687559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7015e3-a7ad-43b3-879d-a3e3574cf242 service placement] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:31:08.691119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b5129-eee6-4694-a3f5-395e4f772111 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:31:08.691742 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b5129-eee6-4694-a3f5-395e4f772111 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:31:08.692022 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b5129-eee6-4694-a3f5-395e4f772111 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:31:08.692290 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b5129-eee6-4694-a3f5-395e4f772111 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:31:08.700432 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b5129-eee6-4694-a3f5-395e4f772111 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:31:08.701849 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b5129-eee6-4694-a3f5-395e4f772111 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:31:08.728806 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 953/1921] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:31:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:31:08.776388 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ca635-e5a7-4ca2-8e1b-678ba0c49773 service nova] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:31:08.786692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ca635-e5a7-4ca2-8e1b-678ba0c49773 service nova] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:31:08.790352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33cc676-f7d8-45bb-b910-8a6285b7017e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:31:08.790955 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cc676-f7d8-45bb-b910-8a6285b7017e service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:31:08.791233 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cc676-f7d8-45bb-b910-8a6285b7017e service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:31:08.791503 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cc676-f7d8-45bb-b910-8a6285b7017e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:31:08.800079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33cc676-f7d8-45bb-b910-8a6285b7017e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:31:08.801498 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33cc676-f7d8-45bb-b910-8a6285b7017e service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:31:08.828952 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 969/1922] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:31:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:31:15.853498 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae98cf2-2305-4a4f-8abd-725ee6336068 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:31:15.854028 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae98cf2-2305-4a4f-8abd-725ee6336068 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:31:15.854140 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae98cf2-2305-4a4f-8abd-725ee6336068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:31:15.893917 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-8ae98cf2-2305-4a4f-8abd-725ee6336068 None None] MFA Rules not processed for user `dc5388b90d0847739330a095f4d5f106`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:31:15.895130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8ae98cf2-2305-4a4f-8abd-725ee6336068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:31:15.902185 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 954/1923] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:31:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:31:16.211132 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33cc676-f7d8-45bb-b910-8a6285b7017e service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:31:16.220682 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33cc676-f7d8-45bb-b910-8a6285b7017e service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:31:16.224204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2688b-a605-48f0-ad74-0fd60a4cad8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:31:16.224794 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2688b-a605-48f0-ad74-0fd60a4cad8d service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:31:16.225069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2688b-a605-48f0-ad74-0fd60a4cad8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:31:16.225338 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2688b-a605-48f0-ad74-0fd60a4cad8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:31:16.227048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ddd2688b-a605-48f0-ad74-0fd60a4cad8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:31:16.248061 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2688b-a605-48f0-ad74-0fd60a4cad8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:31:16.249918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2688b-a605-48f0-ad74-0fd60a4cad8d service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:31:16.257710 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 970/1924] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:31:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:31:16.588717 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae98cf2-2305-4a4f-8abd-725ee6336068 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:31:16.610168 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae98cf2-2305-4a4f-8abd-725ee6336068 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:31:16.616352 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4e524-041e-41e1-bccb-4ef23e23165b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:31:16.617006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4e524-041e-41e1-bccb-4ef23e23165b service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:31:16.617467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4e524-041e-41e1-bccb-4ef23e23165b service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:31:16.617739 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4e524-041e-41e1-bccb-4ef23e23165b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:31:16.619760 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-29f4e524-041e-41e1-bccb-4ef23e23165b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:31:16.649717 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4e524-041e-41e1-bccb-4ef23e23165b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:31:16.651330 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4e524-041e-41e1-bccb-4ef23e23165b service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:31:16.661578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 955/1925] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:31:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:31:25.447551 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2688b-a605-48f0-ad74-0fd60a4cad8d service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:31:25.463839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2688b-a605-48f0-ad74-0fd60a4cad8d service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:31:25.469801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5668c9-0f71-4b84-8416-b5fd0869e318 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:31:25.470406 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5668c9-0f71-4b84-8416-b5fd0869e318 service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:31:25.470686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5668c9-0f71-4b84-8416-b5fd0869e318 service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:31:25.470941 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5668c9-0f71-4b84-8416-b5fd0869e318 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:31:25.472727 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-bf5668c9-0f71-4b84-8416-b5fd0869e318 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:31:25.484827 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5668c9-0f71-4b84-8416-b5fd0869e318 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:31:25.486409 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5668c9-0f71-4b84-8416-b5fd0869e318 service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:31:25.497504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 971/1926] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:31:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:31:26.657244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=49 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:31:29.427223 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4e524-041e-41e1-bccb-4ef23e23165b service placement] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:31:29.439953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4e524-041e-41e1-bccb-4ef23e23165b service placement] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:31:29.445525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676db8f7-717f-4b9d-b196-a2e8fb280bd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:31:29.446129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676db8f7-717f-4b9d-b196-a2e8fb280bd3 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:31:29.446411 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676db8f7-717f-4b9d-b196-a2e8fb280bd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:31:29.446680 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676db8f7-717f-4b9d-b196-a2e8fb280bd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:31:29.448422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-676db8f7-717f-4b9d-b196-a2e8fb280bd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:31:29.461287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676db8f7-717f-4b9d-b196-a2e8fb280bd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:31:29.462867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676db8f7-717f-4b9d-b196-a2e8fb280bd3 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:31:29.474033 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 956/1927] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:31:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:31:35.492684 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=52 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:31:39.469639 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=6 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:31:59.235401 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5668c9-0f71-4b84-8416-b5fd0869e318 service nova] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:31:59.246107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5668c9-0f71-4b84-8416-b5fd0869e318 service nova] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:31:59.251970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc121f-1e59-4bdc-8ffe-f2f476b28c0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:31:59.252592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc121f-1e59-4bdc-8ffe-f2f476b28c0e service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:31:59.252866 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc121f-1e59-4bdc-8ffe-f2f476b28c0e service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:31:59.253134 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc121f-1e59-4bdc-8ffe-f2f476b28c0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:31:59.261427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc121f-1e59-4bdc-8ffe-f2f476b28c0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:31:59.264100 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc121f-1e59-4bdc-8ffe-f2f476b28c0e service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:31:59.292843 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 972/1928] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:31:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:32:03.818439 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676db8f7-717f-4b9d-b196-a2e8fb280bd3 service nova] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:32:03.827543 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676db8f7-717f-4b9d-b196-a2e8fb280bd3 service nova] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:32:03.831100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c68af3-eb90-46e8-83d8-a1ea571efc2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:32:03.831736 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c68af3-eb90-46e8-83d8-a1ea571efc2f service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:32:03.832020 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c68af3-eb90-46e8-83d8-a1ea571efc2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:32:03.832292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c68af3-eb90-46e8-83d8-a1ea571efc2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:32:03.839286 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c68af3-eb90-46e8-83d8-a1ea571efc2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:32:03.840728 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c68af3-eb90-46e8-83d8-a1ea571efc2f service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:32:03.848513 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 957/1929] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:32:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:32:09.290086 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=13 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:32:13.845022 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=6 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:33:12.160143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc121f-1e59-4bdc-8ffe-f2f476b28c0e service nova] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:33:12.179671 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc121f-1e59-4bdc-8ffe-f2f476b28c0e service nova] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:33:12.186484 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad3a67-894a-42a7-a8aa-258f73caa37f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:33:12.187319 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad3a67-894a-42a7-a8aa-258f73caa37f service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:33:12.187734 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad3a67-894a-42a7-a8aa-258f73caa37f service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:33:12.188110 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad3a67-894a-42a7-a8aa-258f73caa37f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:33:12.199699 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad3a67-894a-42a7-a8aa-258f73caa37f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:33:12.201526 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad3a67-894a-42a7-a8aa-258f73caa37f service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:33:12.213756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 973/1930] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:33:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:33:17.818877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c68af3-eb90-46e8-83d8-a1ea571efc2f service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:33:17.888753 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c68af3-eb90-46e8-83d8-a1ea571efc2f service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:33:17.894165 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748f788-9c3d-4b73-8e8b-6d1e4e6a2f5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:33:17.894765 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748f788-9c3d-4b73-8e8b-6d1e4e6a2f5b service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:33:17.895044 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748f788-9c3d-4b73-8e8b-6d1e4e6a2f5b service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:33:17.895314 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748f788-9c3d-4b73-8e8b-6d1e4e6a2f5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:33:17.897400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-5748f788-9c3d-4b73-8e8b-6d1e4e6a2f5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:33:17.974997 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748f788-9c3d-4b73-8e8b-6d1e4e6a2f5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:33:17.976613 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748f788-9c3d-4b73-8e8b-6d1e4e6a2f5b service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:33:17.994362 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 958/1931] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:33:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:33:22.208702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=6 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:33:27.991860 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=69 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:35:09.667612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad3a67-894a-42a7-a8aa-258f73caa37f service nova] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:35:09.721656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad3a67-894a-42a7-a8aa-258f73caa37f service nova] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:35:09.726290 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf3ebe-596b-44be-b742-bbfce555795c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:35:09.726489 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf3ebe-596b-44be-b742-bbfce555795c service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:35:09.726757 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf3ebe-596b-44be-b742-bbfce555795c service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:35:09.727026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf3ebe-596b-44be-b742-bbfce555795c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:35:09.728978 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ecaf3ebe-596b-44be-b742-bbfce555795c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:35:09.815476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf3ebe-596b-44be-b742-bbfce555795c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:35:09.816877 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf3ebe-596b-44be-b742-bbfce555795c service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:35:09.835710 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 974/1932] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:35:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:35:19.832585 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=61 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:35:46.065658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748f788-9c3d-4b73-8e8b-6d1e4e6a2f5b service nova] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:35:46.067218 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-5748f788-9c3d-4b73-8e8b-6d1e4e6a2f5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:35:46.125815 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748f788-9c3d-4b73-8e8b-6d1e4e6a2f5b service nova] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:35:46.126038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321e34d-cd07-4526-8d77-ebfb0169613d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:35:46.126518 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e34d-cd07-4526-8d77-ebfb0169613d service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:35:46.126793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e34d-cd07-4526-8d77-ebfb0169613d service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:35:46.127058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321e34d-cd07-4526-8d77-ebfb0169613d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:35:46.128901 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-f321e34d-cd07-4526-8d77-ebfb0169613d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:35:46.142692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321e34d-cd07-4526-8d77-ebfb0169613d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:35:46.144128 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321e34d-cd07-4526-8d77-ebfb0169613d service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:35:46.155016 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 959/1933] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:35:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:35:47.265246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf3ebe-596b-44be-b742-bbfce555795c service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:35:47.266839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ecaf3ebe-596b-44be-b742-bbfce555795c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:35:47.279060 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf3ebe-596b-44be-b742-bbfce555795c service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:35:47.282558 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7e407-4cea-4bf8-a657-3b46b440ee5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:35:47.283154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7e407-4cea-4bf8-a657-3b46b440ee5e service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:35:47.283433 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7e407-4cea-4bf8-a657-3b46b440ee5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:35:47.283739 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7e407-4cea-4bf8-a657-3b46b440ee5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:35:47.285492 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-cce7e407-4cea-4bf8-a657-3b46b440ee5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:35:47.298298 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7e407-4cea-4bf8-a657-3b46b440ee5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:35:47.299757 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7e407-4cea-4bf8-a657-3b46b440ee5e service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:35:47.310487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 975/1934] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:35:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:35:56.150229 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=26 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:35:57.305596 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=6 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:36:03.144355 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0bc20-4c18-4086-8f35-73013a83210e None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:36:03.144857 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0bc20-4c18-4086-8f35-73013a83210e None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:36:03.144969 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0bc20-4c18-4086-8f35-73013a83210e None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:36:03.147018 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 960/1935] 10.222.0.13 () {54 vars in 961 bytes} [Fri Oct 21 21:36:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 21:36:03.151892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eadd7f-0267-4b05-add7-ebaae2645dac None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:36:03.152328 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eadd7f-0267-4b05-add7-ebaae2645dac None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:36:03.152433 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eadd7f-0267-4b05-add7-ebaae2645dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:36:03.207591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-67eadd7f-0267-4b05-add7-ebaae2645dac None None] MFA Rules not processed for user `124f2b5b95e8450ea6867f6868b9e810`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:36:03.209362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-67eadd7f-0267-4b05-add7-ebaae2645dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:36:03.269536 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 976/1936] 10.222.0.13 () {58 vars in 1058 bytes} [Fri Oct 21 21:36:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:36:03.277863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0bc20-4c18-4086-8f35-73013a83210e None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:36:03.291075 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0bc20-4c18-4086-8f35-73013a83210e None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:36:03.295947 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fca27-2a62-47f7-8543-a058ced052ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:36:03.296532 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fca27-2a62-47f7-8543-a058ced052ad service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:36:03.296806 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fca27-2a62-47f7-8543-a058ced052ad service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:36:03.297350 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fca27-2a62-47f7-8543-a058ced052ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:36:03.299123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-186fca27-2a62-47f7-8543-a058ced052ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:36:03.310422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fca27-2a62-47f7-8543-a058ced052ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:36:03.312007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fca27-2a62-47f7-8543-a058ced052ad service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:36:03.322051 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 961/1937] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:36:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:36:13.261014 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=43 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:36:13.317374 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=6 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:37:06.746141 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eadd7f-0267-4b05-add7-ebaae2645dac None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:37:06.756112 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eadd7f-0267-4b05-add7-ebaae2645dac None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:37:06.759672 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106a5ef-c9cc-41e0-98aa-fffc5edac005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:37:06.760272 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106a5ef-c9cc-41e0-98aa-fffc5edac005 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:37:06.760552 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106a5ef-c9cc-41e0-98aa-fffc5edac005 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:37:06.760822 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106a5ef-c9cc-41e0-98aa-fffc5edac005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:37:06.762631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b106a5ef-c9cc-41e0-98aa-fffc5edac005 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:37:06.773157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106a5ef-c9cc-41e0-98aa-fffc5edac005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:37:06.774582 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106a5ef-c9cc-41e0-98aa-fffc5edac005 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:37:06.782824 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 977/1938] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:37:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:37:10.185027 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fca27-2a62-47f7-8543-a058ced052ad service placement] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:37:10.186626 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-186fca27-2a62-47f7-8543-a058ced052ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:37:10.211011 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fca27-2a62-47f7-8543-a058ced052ad service placement] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:37:10.214591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14066d44-1b6e-4770-abdd-acb06fc06cf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:37:10.215195 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14066d44-1b6e-4770-abdd-acb06fc06cf6 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:37:10.215483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14066d44-1b6e-4770-abdd-acb06fc06cf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:37:10.215778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14066d44-1b6e-4770-abdd-acb06fc06cf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:37:10.217730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-14066d44-1b6e-4770-abdd-acb06fc06cf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:37:10.291082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14066d44-1b6e-4770-abdd-acb06fc06cf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:37:10.292540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14066d44-1b6e-4770-abdd-acb06fc06cf6 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:37:10.302090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 962/1939] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:37:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:37:10.345605 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106a5ef-c9cc-41e0-98aa-fffc5edac005 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:37:10.354586 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106a5ef-c9cc-41e0-98aa-fffc5edac005 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:37:10.358261 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465131f-b4f2-4edb-b40f-b4ac5c33eb93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:37:10.358856 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465131f-b4f2-4edb-b40f-b4ac5c33eb93 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:37:10.359152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465131f-b4f2-4edb-b40f-b4ac5c33eb93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:37:10.359416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465131f-b4f2-4edb-b40f-b4ac5c33eb93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:37:10.361517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-7465131f-b4f2-4edb-b40f-b4ac5c33eb93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:37:10.441231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465131f-b4f2-4edb-b40f-b4ac5c33eb93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:37:10.442750 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465131f-b4f2-4edb-b40f-b4ac5c33eb93 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:37:10.453064 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 978/1940] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:37:10 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:37:10.504358 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14066d44-1b6e-4770-abdd-acb06fc06cf6 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:37:10.513050 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14066d44-1b6e-4770-abdd-acb06fc06cf6 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:37:10.516663 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e78cf0-70ac-4470-acdb-d95ee96487eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:37:10.517264 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e78cf0-70ac-4470-acdb-d95ee96487eb service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:37:10.517540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e78cf0-70ac-4470-acdb-d95ee96487eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:37:10.517815 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e78cf0-70ac-4470-acdb-d95ee96487eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:37:10.519670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c1e78cf0-70ac-4470-acdb-d95ee96487eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:37:10.578317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e78cf0-70ac-4470-acdb-d95ee96487eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:37:10.579865 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e78cf0-70ac-4470-acdb-d95ee96487eb service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:37:10.589367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 963/1941] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:37:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:37:10.672546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465131f-b4f2-4edb-b40f-b4ac5c33eb93 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:37:10.681522 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465131f-b4f2-4edb-b40f-b4ac5c33eb93 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:37:10.685109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e53fb2-219d-4d84-91e0-2c51e85b95c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:37:10.685706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e53fb2-219d-4d84-91e0-2c51e85b95c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:37:10.685984 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e53fb2-219d-4d84-91e0-2c51e85b95c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:37:10.686254 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e53fb2-219d-4d84-91e0-2c51e85b95c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:37:10.688146 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-24e53fb2-219d-4d84-91e0-2c51e85b95c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:37:10.760443 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e53fb2-219d-4d84-91e0-2c51e85b95c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:37:10.761965 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e53fb2-219d-4d84-91e0-2c51e85b95c8 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:37:10.771644 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 979/1942] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:37:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:37:10.962336 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced76ce-74fe-46f5-9913-81c0432da6a5 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:37:10.962625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced76ce-74fe-46f5-9913-81c0432da6a5 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:37:10.962902 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced76ce-74fe-46f5-9913-81c0432da6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:37:11.014714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-dced76ce-74fe-46f5-9913-81c0432da6a5 None None] MFA Rules not processed for user `2346003afe74477496045e9bc72173d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:37:11.015968 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-dced76ce-74fe-46f5-9913-81c0432da6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:37:11.060464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:37:11.079811 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 964/1943] 10.222.0.13 () {58 vars in 967 bytes} [Fri Oct 21 21:37:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:37:11.124941 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e53fb2-219d-4d84-91e0-2c51e85b95c8 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:37:11.126496 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-24e53fb2-219d-4d84-91e0-2c51e85b95c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:37:11.187024 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:37:11.214272 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e53fb2-219d-4d84-91e0-2c51e85b95c8 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:37:11.219465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824af9fd-fcd0-4317-b27e-e325f82dcf49 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ce720c8df014f9a8af062138aecd1b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:37:11.220123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824af9fd-fcd0-4317-b27e-e325f82dcf49 service swift] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:37:11.220396 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824af9fd-fcd0-4317-b27e-e325f82dcf49 service swift] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:37:11.220664 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824af9fd-fcd0-4317-b27e-e325f82dcf49 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:37:11.228588 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824af9fd-fcd0-4317-b27e-e325f82dcf49 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:37:11.230154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824af9fd-fcd0-4317-b27e-e325f82dcf49 service swift] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:37:11.237923 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 980/1944] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Oct 21 21:37:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 21:37:12.473313 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced76ce-74fe-46f5-9913-81c0432da6a5 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:37:12.488103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced76ce-74fe-46f5-9913-81c0432da6a5 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:37:12.498625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908d2bb-5c84-4efc-aaf3-23d965c82f20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:37:12.499197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d2bb-5c84-4efc-aaf3-23d965c82f20 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:37:12.499557 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d2bb-5c84-4efc-aaf3-23d965c82f20 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:37:12.499780 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d2bb-5c84-4efc-aaf3-23d965c82f20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:37:12.510138 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908d2bb-5c84-4efc-aaf3-23d965c82f20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:37:12.512011 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908d2bb-5c84-4efc-aaf3-23d965c82f20 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:37:12.517803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eb9b2-932f-47e5-8e7b-8654f9eaf7aa None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:37:12.518942 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eb9b2-932f-47e5-8e7b-8654f9eaf7aa None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:37:12.519234 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eb9b2-932f-47e5-8e7b-8654f9eaf7aa None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:37:12.521137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 981/1945] 10.222.0.13 () {54 vars in 912 bytes} [Fri Oct 21 21:37:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 21:37:12.523672 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 965/1946] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:37:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:37:12.526405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e937c-cabb-4d21-b4d2-d6c23b02b110 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:37:12.526689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e937c-cabb-4d21-b4d2-d6c23b02b110 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:37:12.527022 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e937c-cabb-4d21-b4d2-d6c23b02b110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:37:12.548573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-840e937c-cabb-4d21-b4d2-d6c23b02b110 None None] MFA Rules not processed for user `124f2b5b95e8450ea6867f6868b9e810`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:37:12.550508 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-840e937c-cabb-4d21-b4d2-d6c23b02b110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:37:12.561707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 982/1947] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Oct 21 21:37:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:37:12.585338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908d2bb-5c84-4efc-aaf3-23d965c82f20 service nova] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:37:12.597671 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908d2bb-5c84-4efc-aaf3-23d965c82f20 service nova] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:37:12.602644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee6aa9-09e3-4d37-ae53-b34202de25ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:37:12.603244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee6aa9-09e3-4d37-ae53-b34202de25ec service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:37:12.603523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee6aa9-09e3-4d37-ae53-b34202de25ec service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:37:12.603826 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee6aa9-09e3-4d37-ae53-b34202de25ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:37:12.612226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee6aa9-09e3-4d37-ae53-b34202de25ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:37:12.613776 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee6aa9-09e3-4d37-ae53-b34202de25ec service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:37:12.623355 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 966/1948] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:37:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:37:14.891124 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13b37a-b6f7-456a-9efd-7f0746c93a31 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:37:14.891651 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13b37a-b6f7-456a-9efd-7f0746c93a31 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:37:14.891767 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13b37a-b6f7-456a-9efd-7f0746c93a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:37:14.919862 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-2e13b37a-b6f7-456a-9efd-7f0746c93a31 None None] MFA Rules not processed for user `64e9cccb41a04aef83ab417783d82fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:37:14.921062 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2e13b37a-b6f7-456a-9efd-7f0746c93a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:37:14.928074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 983/1949] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:37:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:37:15.230555 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee6aa9-09e3-4d37-ae53-b34202de25ec service placement] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:37:15.239713 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee6aa9-09e3-4d37-ae53-b34202de25ec service placement] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:37:15.243224 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168533e-2ac6-4851-a94e-d383301781f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:37:15.243843 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168533e-2ac6-4851-a94e-d383301781f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:37:15.244123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168533e-2ac6-4851-a94e-d383301781f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:37:15.244398 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168533e-2ac6-4851-a94e-d383301781f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:37:15.246088 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c168533e-2ac6-4851-a94e-d383301781f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:37:15.256589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168533e-2ac6-4851-a94e-d383301781f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:37:15.257995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168533e-2ac6-4851-a94e-d383301781f3 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:37:15.265979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 967/1950] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:37:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:37:15.588343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13b37a-b6f7-456a-9efd-7f0746c93a31 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:37:15.589964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2e13b37a-b6f7-456a-9efd-7f0746c93a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:37:15.604400 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13b37a-b6f7-456a-9efd-7f0746c93a31 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:37:15.609289 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb065bb3-4d4c-4f5f-950e-45b7b5349215 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:37:15.609962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb065bb3-4d4c-4f5f-950e-45b7b5349215 service placement] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:37:15.610306 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb065bb3-4d4c-4f5f-950e-45b7b5349215 service placement] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:37:15.610661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb065bb3-4d4c-4f5f-950e-45b7b5349215 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:37:15.619738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb065bb3-4d4c-4f5f-950e-45b7b5349215 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:37:15.621297 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb065bb3-4d4c-4f5f-950e-45b7b5349215 service placement] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:37:15.632106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 984/1951] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:37:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:37:25.248955 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168533e-2ac6-4851-a94e-d383301781f3 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:37:25.251687 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c168533e-2ac6-4851-a94e-d383301781f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:37:25.269525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168533e-2ac6-4851-a94e-d383301781f3 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:37:25.275260 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fc21d-27c5-4b9f-889d-c5823b82f15d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:37:25.275904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fc21d-27c5-4b9f-889d-c5823b82f15d service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:37:25.276182 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fc21d-27c5-4b9f-889d-c5823b82f15d service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:37:25.276451 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fc21d-27c5-4b9f-889d-c5823b82f15d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:37:25.286419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fc21d-27c5-4b9f-889d-c5823b82f15d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:37:25.288030 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fc21d-27c5-4b9f-889d-c5823b82f15d service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:37:25.300274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 968/1952] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:37:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:37:25.626892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=28 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:37:35.294770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=30 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:38:17.319522 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb065bb3-4d4c-4f5f-950e-45b7b5349215 service placement] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:38:17.321103 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-cb065bb3-4d4c-4f5f-950e-45b7b5349215 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:38:17.333180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb065bb3-4d4c-4f5f-950e-45b7b5349215 service placement] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:38:17.336657 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b075eda-0c69-42d2-8207-dc8ec930298f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:38:17.337245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b075eda-0c69-42d2-8207-dc8ec930298f service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:38:17.337520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b075eda-0c69-42d2-8207-dc8ec930298f service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:38:17.337787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b075eda-0c69-42d2-8207-dc8ec930298f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:38:17.345670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b075eda-0c69-42d2-8207-dc8ec930298f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:38:17.347773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b075eda-0c69-42d2-8207-dc8ec930298f service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:38:17.356447 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 985/1953] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:38:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:38:27.352442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=6 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:39:15.935218 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fc21d-27c5-4b9f-889d-c5823b82f15d service nova] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:39:15.966897 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fc21d-27c5-4b9f-889d-c5823b82f15d service nova] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:39:15.973105 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d100642-0589-4382-936b-7d0fe6c6615b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:39:15.973821 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d100642-0589-4382-936b-7d0fe6c6615b service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:39:15.974288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d100642-0589-4382-936b-7d0fe6c6615b service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:39:15.974790 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d100642-0589-4382-936b-7d0fe6c6615b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:39:15.977204 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1d100642-0589-4382-936b-7d0fe6c6615b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:39:15.992675 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d100642-0589-4382-936b-7d0fe6c6615b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:39:15.995314 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d100642-0589-4382-936b-7d0fe6c6615b service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:39:16.011303 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 969/1954] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:39:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:39:26.005884 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=12 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:40:10.942249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b075eda-0c69-42d2-8207-dc8ec930298f service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:40:10.968347 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b075eda-0c69-42d2-8207-dc8ec930298f service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:40:10.974066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c818405c-35c1-473b-b330-18c5e2c3312a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:40:10.974706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818405c-35c1-473b-b330-18c5e2c3312a service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:40:10.974999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818405c-35c1-473b-b330-18c5e2c3312a service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:40:10.975262 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818405c-35c1-473b-b330-18c5e2c3312a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:40:10.983557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c818405c-35c1-473b-b330-18c5e2c3312a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:40:10.985159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c818405c-35c1-473b-b330-18c5e2c3312a service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:40:10.994637 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 986/1955] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:40:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:40:20.989984 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=12 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:40:46.250979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d100642-0589-4382-936b-7d0fe6c6615b service nova] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:40:46.288028 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d100642-0589-4382-936b-7d0fe6c6615b service nova] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:40:46.291551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d50e9-0efb-4aaa-8384-d264e6fe8476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:40:46.292168 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d50e9-0efb-4aaa-8384-d264e6fe8476 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:40:46.292448 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d50e9-0efb-4aaa-8384-d264e6fe8476 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:40:46.292720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d50e9-0efb-4aaa-8384-d264e6fe8476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:40:46.316319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d50e9-0efb-4aaa-8384-d264e6fe8476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:40:46.317729 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d50e9-0efb-4aaa-8384-d264e6fe8476 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:40:46.339213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 970/1956] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:40:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:40:47.188783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c818405c-35c1-473b-b330-18c5e2c3312a service nova] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:40:47.207765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c818405c-35c1-473b-b330-18c5e2c3312a service nova] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:40:47.211304 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750dfc6-e7cd-4350-aed8-fa98a6e0ae3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:40:47.211927 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750dfc6-e7cd-4350-aed8-fa98a6e0ae3c service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:40:47.212206 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750dfc6-e7cd-4350-aed8-fa98a6e0ae3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:40:47.212479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750dfc6-e7cd-4350-aed8-fa98a6e0ae3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:40:47.223146 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750dfc6-e7cd-4350-aed8-fa98a6e0ae3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:40:47.224601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750dfc6-e7cd-4350-aed8-fa98a6e0ae3c service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:40:47.235215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 987/1957] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:40:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:40:56.336660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=28 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:40:57.230517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=11 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:41:12.143387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d50e9-0efb-4aaa-8384-d264e6fe8476 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:41:12.164964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d50e9-0efb-4aaa-8384-d264e6fe8476 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:41:12.169887 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab81f8-578f-4b2e-bc6a-238d40da4e8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:41:12.170482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab81f8-578f-4b2e-bc6a-238d40da4e8e service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:41:12.170758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab81f8-578f-4b2e-bc6a-238d40da4e8e service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:41:12.171029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab81f8-578f-4b2e-bc6a-238d40da4e8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:41:12.179345 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab81f8-578f-4b2e-bc6a-238d40da4e8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:41:12.180931 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab81f8-578f-4b2e-bc6a-238d40da4e8e service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:41:12.190786 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 971/1958] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:41:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:41:22.186483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=10 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:42:09.473657 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750dfc6-e7cd-4350-aed8-fa98a6e0ae3c service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:42:09.475909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0750dfc6-e7cd-4350-aed8-fa98a6e0ae3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:42:09.511642 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750dfc6-e7cd-4350-aed8-fa98a6e0ae3c service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:42:09.517054 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe93edd-d578-46b0-9fd6-648bf4e50334 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:42:09.517653 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe93edd-d578-46b0-9fd6-648bf4e50334 service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:42:09.517931 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe93edd-d578-46b0-9fd6-648bf4e50334 service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:42:09.518201 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe93edd-d578-46b0-9fd6-648bf4e50334 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:42:09.526708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe93edd-d578-46b0-9fd6-648bf4e50334 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:42:09.528337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe93edd-d578-46b0-9fd6-648bf4e50334 service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:42:09.553837 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 988/1959] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:42:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:42:12.625425 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab81f8-578f-4b2e-bc6a-238d40da4e8e service placement] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:42:12.634631 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab81f8-578f-4b2e-bc6a-238d40da4e8e service placement] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:42:12.638153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ef4d3-b627-46de-9c14-e28b81574891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:42:12.638745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ef4d3-b627-46de-9c14-e28b81574891 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:42:12.639025 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ef4d3-b627-46de-9c14-e28b81574891 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:42:12.639299 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ef4d3-b627-46de-9c14-e28b81574891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:42:12.646661 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ef4d3-b627-46de-9c14-e28b81574891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:42:12.648106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ef4d3-b627-46de-9c14-e28b81574891 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:42:12.655846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 972/1960] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:42:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:42:19.551256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=22 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:42:22.652388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=6 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:43:14.698878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe93edd-d578-46b0-9fd6-648bf4e50334 service nova] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:43:14.717682 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe93edd-d578-46b0-9fd6-648bf4e50334 service nova] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:43:14.721234 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5928279-eb97-4904-b8a9-73c0ff349a4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:43:14.721832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5928279-eb97-4904-b8a9-73c0ff349a4f service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:43:14.722107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5928279-eb97-4904-b8a9-73c0ff349a4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:43:14.722378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5928279-eb97-4904-b8a9-73c0ff349a4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:43:14.730792 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5928279-eb97-4904-b8a9-73c0ff349a4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:43:14.732286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5928279-eb97-4904-b8a9-73c0ff349a4f service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:43:14.759400 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 989/1961] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:43:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:43:14.801667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ef4d3-b627-46de-9c14-e28b81574891 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:43:14.811188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ef4d3-b627-46de-9c14-e28b81574891 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:43:14.814655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aad310-d4db-4264-bdb4-161e71095459 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:43:14.815235 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aad310-d4db-4264-bdb4-161e71095459 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:43:14.815508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aad310-d4db-4264-bdb4-161e71095459 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:43:14.815812 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aad310-d4db-4264-bdb4-161e71095459 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:43:14.824310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aad310-d4db-4264-bdb4-161e71095459 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:43:14.826347 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aad310-d4db-4264-bdb4-161e71095459 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:43:14.852538 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 973/1962] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:43:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:43:14.895686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5928279-eb97-4904-b8a9-73c0ff349a4f service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:43:14.904083 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5928279-eb97-4904-b8a9-73c0ff349a4f service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:43:14.907678 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99589ed-65fe-4497-98f2-b1caf5a2ec17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:43:14.908274 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99589ed-65fe-4497-98f2-b1caf5a2ec17 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:43:14.908547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99589ed-65fe-4497-98f2-b1caf5a2ec17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:43:14.908816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99589ed-65fe-4497-98f2-b1caf5a2ec17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:43:14.917338 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99589ed-65fe-4497-98f2-b1caf5a2ec17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:43:14.918770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99589ed-65fe-4497-98f2-b1caf5a2ec17 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:43:14.946224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 990/1963] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:43:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:43:15.429102 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aad310-d4db-4264-bdb4-161e71095459 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:43:15.438491 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aad310-d4db-4264-bdb4-161e71095459 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:43:15.442164 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3ded0-8c06-4fa0-885f-68274bb1fb59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:43:15.442777 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3ded0-8c06-4fa0-885f-68274bb1fb59 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:43:15.443058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3ded0-8c06-4fa0-885f-68274bb1fb59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:43:15.443331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3ded0-8c06-4fa0-885f-68274bb1fb59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:43:15.445145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-54a3ded0-8c06-4fa0-885f-68274bb1fb59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:43:15.527386 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3ded0-8c06-4fa0-885f-68274bb1fb59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:43:15.529370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3ded0-8c06-4fa0-885f-68274bb1fb59 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:43:15.548339 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 974/1964] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:43:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:43:18.427090 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99589ed-65fe-4497-98f2-b1caf5a2ec17 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:43:18.449036 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99589ed-65fe-4497-98f2-b1caf5a2ec17 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:43:18.455905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca367d87-1439-429f-ab32-8750621e80df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:43:18.456586 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca367d87-1439-429f-ab32-8750621e80df service placement] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:43:18.456868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca367d87-1439-429f-ab32-8750621e80df service placement] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:43:18.457143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca367d87-1439-429f-ab32-8750621e80df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:43:18.458905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ca367d87-1439-429f-ab32-8750621e80df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:43:18.484939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca367d87-1439-429f-ab32-8750621e80df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:43:18.487156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca367d87-1439-429f-ab32-8750621e80df service placement] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:43:18.497998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 991/1965] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:43:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:43:22.110169 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3ded0-8c06-4fa0-885f-68274bb1fb59 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:43:22.112226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-54a3ded0-8c06-4fa0-885f-68274bb1fb59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:43:22.126082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3ded0-8c06-4fa0-885f-68274bb1fb59 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:43:22.130640 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d539b80-b55b-4e86-8380-0d237172765b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:43:22.131262 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d539b80-b55b-4e86-8380-0d237172765b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:43:22.131538 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d539b80-b55b-4e86-8380-0d237172765b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:43:22.131843 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d539b80-b55b-4e86-8380-0d237172765b admin admin] PATH_INFO: `/v3/users/07447907733f47a08c0a6554035058dc` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:43:22.145998 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d539b80-b55b-4e86-8380-0d237172765b admin admin] RBAC: Authorizing `identity:delete_user(user_id=07447907733f47a08c0a6554035058dc)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:43:22.147961 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d539b80-b55b-4e86-8380-0d237172765b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:43:22.191553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-4d539b80-b55b-4e86-8380-0d237172765b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07447907733f47a08c0a6554035058dc'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:22.192431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-4d539b80-b55b-4e86-8380-0d237172765b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07447907733f47a08c0a6554035058dc'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:22.204905 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-4d539b80-b55b-4e86-8380-0d237172765b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07447907733f47a08c0a6554035058dc'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:22.212109 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-4d539b80-b55b-4e86-8380-0d237172765b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07447907733f47a08c0a6554035058dc'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:22.212659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-4d539b80-b55b-4e86-8380-0d237172765b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07447907733f47a08c0a6554035058dc'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:22.220286 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:43:22.225841 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 975/1966] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:43:22 2022] DELETE /identity/v3/users/07447907733f47a08c0a6554035058dc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:43:22.395516 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca367d87-1439-429f-ab32-8750621e80df service placement] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:43:22.406411 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca367d87-1439-429f-ab32-8750621e80df service placement] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:43:22.462308 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12275e-579c-4047-bab3-4bc0c2d17daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:43:22.462947 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12275e-579c-4047-bab3-4bc0c2d17daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:43:22.463227 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12275e-579c-4047-bab3-4bc0c2d17daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:43:22.463498 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12275e-579c-4047-bab3-4bc0c2d17daa admin admin] PATH_INFO: `/v3/projects/434d61a7b1844a5180a564007c8b369f` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:43:22.473898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12275e-579c-4047-bab3-4bc0c2d17daa admin admin] RBAC: Authorizing `identity:delete_project(project_id=434d61a7b1844a5180a564007c8b369f)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:43:22.475861 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12275e-579c-4047-bab3-4bc0c2d17daa admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:43:22.524040 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7e12275e-579c-4047-bab3-4bc0c2d17daa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '434d61a7b1844a5180a564007c8b369f', 'request_id': 'req-7e12275e-579c-4047-bab3-4bc0c2d17daa'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:22.531292 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7e12275e-579c-4047-bab3-4bc0c2d17daa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '434d61a7b1844a5180a564007c8b369f', 'request_id': 'req-7e12275e-579c-4047-bab3-4bc0c2d17daa'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:22.538476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7e12275e-579c-4047-bab3-4bc0c2d17daa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '434d61a7b1844a5180a564007c8b369f', 'request_id': 'req-7e12275e-579c-4047-bab3-4bc0c2d17daa'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:22.540398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:43:22.551806 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7e12275e-579c-4047-bab3-4bc0c2d17daa admin admin] The token cache is being invalidate because project 434d61a7b1844a5180a564007c8b369f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:43:22.552098 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7e12275e-579c-4047-bab3-4bc0c2d17daa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:22.552730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7e12275e-579c-4047-bab3-4bc0c2d17daa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:22.554256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 992/1967] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:43:22 2022] DELETE /identity/v3/projects/434d61a7b1844a5180a564007c8b369f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:43:22.558127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d539b80-b55b-4e86-8380-0d237172765b admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:43:22.560129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-4d539b80-b55b-4e86-8380-0d237172765b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:43:22.573385 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d539b80-b55b-4e86-8380-0d237172765b admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:43:22.628698 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8836cb7-9f7e-49b0-96f1-211850dc5a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:43:22.629319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8836cb7-9f7e-49b0-96f1-211850dc5a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:43:22.629595 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8836cb7-9f7e-49b0-96f1-211850dc5a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:43:22.629860 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8836cb7-9f7e-49b0-96f1-211850dc5a96 admin admin] PATH_INFO: `/v3/users/b948381f46104c1aa3ef0faceab9b444` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:43:22.644374 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8836cb7-9f7e-49b0-96f1-211850dc5a96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b948381f46104c1aa3ef0faceab9b444)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:43:22.646229 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8836cb7-9f7e-49b0-96f1-211850dc5a96 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:43:22.689900 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c8836cb7-9f7e-49b0-96f1-211850dc5a96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b948381f46104c1aa3ef0faceab9b444'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:22.690662 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c8836cb7-9f7e-49b0-96f1-211850dc5a96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b948381f46104c1aa3ef0faceab9b444'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:22.703378 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c8836cb7-9f7e-49b0-96f1-211850dc5a96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b948381f46104c1aa3ef0faceab9b444'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:22.709918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c8836cb7-9f7e-49b0-96f1-211850dc5a96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b948381f46104c1aa3ef0faceab9b444'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:22.710700 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c8836cb7-9f7e-49b0-96f1-211850dc5a96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b948381f46104c1aa3ef0faceab9b444'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:22.723256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 976/1968] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:43:22 2022] DELETE /identity/v3/users/b948381f46104c1aa3ef0faceab9b444 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:43:22.907315 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12275e-579c-4047-bab3-4bc0c2d17daa admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:43:22.908859 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-7e12275e-579c-4047-bab3-4bc0c2d17daa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:43:22.921746 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12275e-579c-4047-bab3-4bc0c2d17daa admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:43:22.979089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e92444-58b6-475b-81dc-a4df46b82874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:43:22.979749 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e92444-58b6-475b-81dc-a4df46b82874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:43:22.980021 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e92444-58b6-475b-81dc-a4df46b82874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:43:22.980285 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e92444-58b6-475b-81dc-a4df46b82874 admin admin] PATH_INFO: `/v3/projects/7cdc3688c24547c6a97b874735746d5e` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:43:22.990248 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e92444-58b6-475b-81dc-a4df46b82874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cdc3688c24547c6a97b874735746d5e)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:43:22.992307 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e92444-58b6-475b-81dc-a4df46b82874 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:43:23.037157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d1e92444-58b6-475b-81dc-a4df46b82874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cdc3688c24547c6a97b874735746d5e', 'request_id': 'req-d1e92444-58b6-475b-81dc-a4df46b82874'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:23.043962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d1e92444-58b6-475b-81dc-a4df46b82874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cdc3688c24547c6a97b874735746d5e', 'request_id': 'req-d1e92444-58b6-475b-81dc-a4df46b82874'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:23.051060 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d1e92444-58b6-475b-81dc-a4df46b82874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cdc3688c24547c6a97b874735746d5e', 'request_id': 'req-d1e92444-58b6-475b-81dc-a4df46b82874'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:23.061594 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d1e92444-58b6-475b-81dc-a4df46b82874 admin admin] The token cache is being invalidate because project 7cdc3688c24547c6a97b874735746d5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:43:23.061881 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d1e92444-58b6-475b-81dc-a4df46b82874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:23.062506 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d1e92444-58b6-475b-81dc-a4df46b82874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:43:23.064036 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 993/1969] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:43:22 2022] DELETE /identity/v3/projects/7cdc3688c24547c6a97b874735746d5e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:43:32.719780 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=59,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:43:33.058113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=58,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:45:12.021366 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5d586-09a4-46db-b825-c7b1e16077ef None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:45:12.021908 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5d586-09a4-46db-b825-c7b1e16077ef None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:45:12.022017 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5d586-09a4-46db-b825-c7b1e16077ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:45:12.049214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-3ff5d586-09a4-46db-b825-c7b1e16077ef None None] MFA Rules not processed for user `00374760bfa648e48dedae8aadb61c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:45:12.050399 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3ff5d586-09a4-46db-b825-c7b1e16077ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:45:12.102647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 977/1970] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:45:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:45:12.403614 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e92444-58b6-475b-81dc-a4df46b82874 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:45:12.405135 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d1e92444-58b6-475b-81dc-a4df46b82874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:45:12.470585 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e92444-58b6-475b-81dc-a4df46b82874 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:45:12.474079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d029ef-da7b-478f-8707-3375b0d7b332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:45:12.474671 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d029ef-da7b-478f-8707-3375b0d7b332 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:45:12.474942 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d029ef-da7b-478f-8707-3375b0d7b332 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:45:12.475208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d029ef-da7b-478f-8707-3375b0d7b332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:45:12.476919 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-25d029ef-da7b-478f-8707-3375b0d7b332 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:45:12.486028 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d029ef-da7b-478f-8707-3375b0d7b332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:45:12.487438 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d029ef-da7b-478f-8707-3375b0d7b332 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:45:12.495655 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 994/1971] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:45:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:45:12.824909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5d586-09a4-46db-b825-c7b1e16077ef None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:45:12.826263 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3ff5d586-09a4-46db-b825-c7b1e16077ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:45:12.909444 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5d586-09a4-46db-b825-c7b1e16077ef None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:45:12.914605 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64512507-ad4f-4732-8837-4e1d3b4d93b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:45:12.915214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64512507-ad4f-4732-8837-4e1d3b4d93b9 service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:45:12.915494 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64512507-ad4f-4732-8837-4e1d3b4d93b9 service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:45:12.915806 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64512507-ad4f-4732-8837-4e1d3b4d93b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:45:12.918147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-64512507-ad4f-4732-8837-4e1d3b4d93b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:45:12.929393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64512507-ad4f-4732-8837-4e1d3b4d93b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:45:12.930984 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64512507-ad4f-4732-8837-4e1d3b4d93b9 service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:45:12.941513 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 978/1972] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:45:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:45:15.330592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d029ef-da7b-478f-8707-3375b0d7b332 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:45:15.332231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-25d029ef-da7b-478f-8707-3375b0d7b332 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:45:15.390759 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d029ef-da7b-478f-8707-3375b0d7b332 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:45:15.394728 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf4ffd-764b-44e2-a997-fff869fee853 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:45:15.395338 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf4ffd-764b-44e2-a997-fff869fee853 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:45:15.395639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf4ffd-764b-44e2-a997-fff869fee853 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:45:15.395918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf4ffd-764b-44e2-a997-fff869fee853 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:45:15.397901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-5ddf4ffd-764b-44e2-a997-fff869fee853 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:45:15.487828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf4ffd-764b-44e2-a997-fff869fee853 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:45:15.489534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf4ffd-764b-44e2-a997-fff869fee853 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:45:15.511238 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 995/1973] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:45:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:45:16.152311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64512507-ad4f-4732-8837-4e1d3b4d93b9 service placement] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:45:16.154373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-64512507-ad4f-4732-8837-4e1d3b4d93b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:45:16.247727 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64512507-ad4f-4732-8837-4e1d3b4d93b9 service placement] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:45:16.253300 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7baad-b2e4-4385-aef6-7a003daf203a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:45:16.253913 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7baad-b2e4-4385-aef6-7a003daf203a service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:45:16.254395 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7baad-b2e4-4385-aef6-7a003daf203a service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:45:16.254833 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7baad-b2e4-4385-aef6-7a003daf203a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:45:16.257200 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-cde7baad-b2e4-4385-aef6-7a003daf203a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:45:16.270306 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7baad-b2e4-4385-aef6-7a003daf203a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:45:16.271936 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7baad-b2e4-4385-aef6-7a003daf203a service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:45:16.286799 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 979/1974] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:45:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:45:21.433349 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf4ffd-764b-44e2-a997-fff869fee853 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:45:21.434807 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-5ddf4ffd-764b-44e2-a997-fff869fee853 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:45:21.451498 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf4ffd-764b-44e2-a997-fff869fee853 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:45:21.456928 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76830ab8-1838-4218-ba19-f6711388a0f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:45:21.457640 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76830ab8-1838-4218-ba19-f6711388a0f1 service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:45:21.457922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76830ab8-1838-4218-ba19-f6711388a0f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:45:21.458190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76830ab8-1838-4218-ba19-f6711388a0f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:45:21.459911 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-76830ab8-1838-4218-ba19-f6711388a0f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:45:21.472405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76830ab8-1838-4218-ba19-f6711388a0f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:45:21.473975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76830ab8-1838-4218-ba19-f6711388a0f1 service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:45:21.485186 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 996/1975] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:45:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:45:26.278754 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=94 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:45:31.480602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=98 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:46:15.117304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7baad-b2e4-4385-aef6-7a003daf203a service nova] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:46:15.118727 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-cde7baad-b2e4-4385-aef6-7a003daf203a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:46:15.133015 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7baad-b2e4-4385-aef6-7a003daf203a service nova] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:46:15.137925 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af616178-31c7-4056-a36f-d81f855e3491 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:46:15.138516 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af616178-31c7-4056-a36f-d81f855e3491 service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:46:15.138795 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af616178-31c7-4056-a36f-d81f855e3491 service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:46:15.139091 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af616178-31c7-4056-a36f-d81f855e3491 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:46:15.140731 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-af616178-31c7-4056-a36f-d81f855e3491 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:46:15.152319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af616178-31c7-4056-a36f-d81f855e3491 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:46:15.153892 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af616178-31c7-4056-a36f-d81f855e3491 service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:46:15.163615 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 980/1976] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:46:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:46:25.159538 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=6 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:47:16.117034 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76830ab8-1838-4218-ba19-f6711388a0f1 service nova] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:16.134888 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76830ab8-1838-4218-ba19-f6711388a0f1 service nova] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:16.138622 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83109cc-7651-4e0c-aa03-8e0692ddc3f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:16.139222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83109cc-7651-4e0c-aa03-8e0692ddc3f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:16.139494 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83109cc-7651-4e0c-aa03-8e0692ddc3f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:16.139811 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83109cc-7651-4e0c-aa03-8e0692ddc3f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:16.141735 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f83109cc-7651-4e0c-aa03-8e0692ddc3f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:47:16.166041 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83109cc-7651-4e0c-aa03-8e0692ddc3f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:16.167675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83109cc-7651-4e0c-aa03-8e0692ddc3f5 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:16.177078 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 997/1977] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:47:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:47:16.265330 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af616178-31c7-4056-a36f-d81f855e3491 service placement] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:16.267011 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-af616178-31c7-4056-a36f-d81f855e3491 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:47:16.277988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af616178-31c7-4056-a36f-d81f855e3491 service placement] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:16.281706 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ab460-d353-46ff-9c6f-014570c147b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:16.282338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ab460-d353-46ff-9c6f-014570c147b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:16.282616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ab460-d353-46ff-9c6f-014570c147b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:16.282889 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ab460-d353-46ff-9c6f-014570c147b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:16.285082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-279ab460-d353-46ff-9c6f-014570c147b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:47:16.298629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ab460-d353-46ff-9c6f-014570c147b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:16.300631 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ab460-d353-46ff-9c6f-014570c147b4 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:16.312205 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 981/1978] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:47:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:47:17.500875 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83109cc-7651-4e0c-aa03-8e0692ddc3f5 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:17.510931 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83109cc-7651-4e0c-aa03-8e0692ddc3f5 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:17.514599 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8003eaf-644f-4983-ac28-a3dca8630fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:17.515198 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8003eaf-644f-4983-ac28-a3dca8630fda service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:17.515473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8003eaf-644f-4983-ac28-a3dca8630fda service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:17.516029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8003eaf-644f-4983-ac28-a3dca8630fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:17.517760 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e8003eaf-644f-4983-ac28-a3dca8630fda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:47:17.530942 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8003eaf-644f-4983-ac28-a3dca8630fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:17.532394 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8003eaf-644f-4983-ac28-a3dca8630fda service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:17.543658 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 998/1979] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:47:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:47:21.522127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ab460-d353-46ff-9c6f-014570c147b4 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:21.530654 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ab460-d353-46ff-9c6f-014570c147b4 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:21.534807 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69f3a4-419b-41e7-b0ed-764acf435b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:21.535395 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69f3a4-419b-41e7-b0ed-764acf435b56 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:21.535707 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69f3a4-419b-41e7-b0ed-764acf435b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:21.535982 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69f3a4-419b-41e7-b0ed-764acf435b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:21.537674 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ec69f3a4-419b-41e7-b0ed-764acf435b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:47:21.598825 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec69f3a4-419b-41e7-b0ed-764acf435b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:21.600290 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec69f3a4-419b-41e7-b0ed-764acf435b56 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:21.610280 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 982/1980] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:47:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:47:24.757503 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8003eaf-644f-4983-ac28-a3dca8630fda service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:24.759047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e8003eaf-644f-4983-ac28-a3dca8630fda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:47:24.773531 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8003eaf-644f-4983-ac28-a3dca8630fda service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:24.777956 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a801e-7254-403f-a706-d21940abd77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:24.778574 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a801e-7254-403f-a706-d21940abd77d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:24.778850 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a801e-7254-403f-a706-d21940abd77d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:24.779118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a801e-7254-403f-a706-d21940abd77d admin admin] PATH_INFO: `/v3/users/e550c842cdbd4da396045c50a6d38866` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:24.793203 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a801e-7254-403f-a706-d21940abd77d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e550c842cdbd4da396045c50a6d38866)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:24.795176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a801e-7254-403f-a706-d21940abd77d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:24.837278 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2f8a801e-7254-403f-a706-d21940abd77d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e550c842cdbd4da396045c50a6d38866'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:47:24.849321 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2f8a801e-7254-403f-a706-d21940abd77d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e550c842cdbd4da396045c50a6d38866'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:47:24.856321 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2f8a801e-7254-403f-a706-d21940abd77d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e550c842cdbd4da396045c50a6d38866'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:47:24.856870 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2f8a801e-7254-403f-a706-d21940abd77d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e550c842cdbd4da396045c50a6d38866'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:47:24.863622 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2f8a801e-7254-403f-a706-d21940abd77d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e550c842cdbd4da396045c50a6d38866'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:47:24.872566 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 999/1981] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:47:24 2022] DELETE /identity/v3/users/e550c842cdbd4da396045c50a6d38866 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:47:25.046768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69f3a4-419b-41e7-b0ed-764acf435b56 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:25.057677 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69f3a4-419b-41e7-b0ed-764acf435b56 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:25.118408 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc08907-9b6b-419b-b1d1-f417a5377d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:25.119038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08907-9b6b-419b-b1d1-f417a5377d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:25.119312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08907-9b6b-419b-b1d1-f417a5377d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:25.119594 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08907-9b6b-419b-b1d1-f417a5377d88 admin admin] PATH_INFO: `/v3/projects/e00d05e0e86e4553ac0a953949538ce1` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:25.120696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc08907-9b6b-419b-b1d1-f417a5377d88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e00d05e0e86e4553ac0a953949538ce1)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:25.122563 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc08907-9b6b-419b-b1d1-f417a5377d88 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:25.167672 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1fc08907-9b6b-419b-b1d1-f417a5377d88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e00d05e0e86e4553ac0a953949538ce1', 'request_id': 'req-1fc08907-9b6b-419b-b1d1-f417a5377d88'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:47:25.174968 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1fc08907-9b6b-419b-b1d1-f417a5377d88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e00d05e0e86e4553ac0a953949538ce1', 'request_id': 'req-1fc08907-9b6b-419b-b1d1-f417a5377d88'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:47:25.180656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1fc08907-9b6b-419b-b1d1-f417a5377d88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e00d05e0e86e4553ac0a953949538ce1', 'request_id': 'req-1fc08907-9b6b-419b-b1d1-f417a5377d88'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:47:25.192081 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1fc08907-9b6b-419b-b1d1-f417a5377d88 admin admin] The token cache is being invalidate because project e00d05e0e86e4553ac0a953949538ce1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:47:25.192365 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1fc08907-9b6b-419b-b1d1-f417a5377d88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:47:25.192988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1fc08907-9b6b-419b-b1d1-f417a5377d88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:47:25.194983 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 983/1982] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:47:25 2022] DELETE /identity/v3/projects/e00d05e0e86e4553ac0a953949538ce1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:47:25.201273 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c7452-bf59-4ccd-a701-194dcc1cba84 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:25.201559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c7452-bf59-4ccd-a701-194dcc1cba84 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:25.201833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c7452-bf59-4ccd-a701-194dcc1cba84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:25.259173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-578c7452-bf59-4ccd-a701-194dcc1cba84 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:47:25.260425 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-578c7452-bf59-4ccd-a701-194dcc1cba84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:47:25.305915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:47:25.318972 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1000/1983] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:47:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:47:25.326630 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3ed31-72c9-464d-9dbb-a6bd3e417495 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:25.326916 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3ed31-72c9-464d-9dbb-a6bd3e417495 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:25.327191 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3ed31-72c9-464d-9dbb-a6bd3e417495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:25.347049 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-75c3ed31-72c9-464d-9dbb-a6bd3e417495 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:47:25.348270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-75c3ed31-72c9-464d-9dbb-a6bd3e417495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:47:25.356127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 984/1984] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:47:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:47:25.360211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c7452-bf59-4ccd-a701-194dcc1cba84 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:25.370096 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c7452-bf59-4ccd-a701-194dcc1cba84 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:25.374526 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f64426-7ce4-41f5-8854-e9abf2144e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:25.375114 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f64426-7ce4-41f5-8854-e9abf2144e8b admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:25.375388 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f64426-7ce4-41f5-8854-e9abf2144e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:25.375717 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f64426-7ce4-41f5-8854-e9abf2144e8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:25.376349 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f64426-7ce4-41f5-8854-e9abf2144e8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:47:25.376647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f64426-7ce4-41f5-8854-e9abf2144e8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:25.378230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f64426-7ce4-41f5-8854-e9abf2144e8b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:25.389256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1001/1985] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:47:25.393567 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3ed31-72c9-464d-9dbb-a6bd3e417495 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:25.399017 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:47:25.402703 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3ed31-72c9-464d-9dbb-a6bd3e417495 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:25.408756 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d4626e-d18e-460f-a2bd-1b95257a8634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:25.409499 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4626e-d18e-460f-a2bd-1b95257a8634 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:25.409783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4626e-d18e-460f-a2bd-1b95257a8634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:25.410217 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4626e-d18e-460f-a2bd-1b95257a8634 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:25.411013 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d4626e-d18e-460f-a2bd-1b95257a8634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:25.412921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d4626e-d18e-460f-a2bd-1b95257a8634 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:25.451890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 985/1986] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:47:25 2022] POST /identity/v3/projects => generated 403 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:47:25.456523 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f64426-7ce4-41f5-8854-e9abf2144e8b admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:25.466267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f64426-7ce4-41f5-8854-e9abf2144e8b admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:25.520271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494659e-dd83-4e07-b49e-85b33c7b5135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:25.520868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494659e-dd83-4e07-b49e-85b33c7b5135 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:25.521152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494659e-dd83-4e07-b49e-85b33c7b5135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:25.521420 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494659e-dd83-4e07-b49e-85b33c7b5135 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:25.522034 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494659e-dd83-4e07-b49e-85b33c7b5135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:25.523989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494659e-dd83-4e07-b49e-85b33c7b5135 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:25.545857 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1002/1987] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:47:25 2022] POST /identity/v3/users => generated 302 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:47:25.550118 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d4626e-d18e-460f-a2bd-1b95257a8634 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:25.560365 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d4626e-d18e-460f-a2bd-1b95257a8634 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:25.564875 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ca4ef-7f01-40b7-964f-796baee7620f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:25.565469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ca4ef-7f01-40b7-964f-796baee7620f admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:25.565749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ca4ef-7f01-40b7-964f-796baee7620f admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:25.566022 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ca4ef-7f01-40b7-964f-796baee7620f admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:25.566655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ca4ef-7f01-40b7-964f-796baee7620f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:25.568287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ca4ef-7f01-40b7-964f-796baee7620f admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:25.578641 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 986/1988] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:47:25.582299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494659e-dd83-4e07-b49e-85b33c7b5135 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:25.592087 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494659e-dd83-4e07-b49e-85b33c7b5135 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:25.597259 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9ba6c-e510-4df0-a774-534cc1cc2c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:25.597845 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9ba6c-e510-4df0-a774-534cc1cc2c1c admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:25.598121 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9ba6c-e510-4df0-a774-534cc1cc2c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:25.598391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9ba6c-e510-4df0-a774-534cc1cc2c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:25.599020 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9ba6c-e510-4df0-a774-534cc1cc2c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:25.600631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9ba6c-e510-4df0-a774-534cc1cc2c1c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:25.610887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1003/1989] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:47:25.614917 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ca4ef-7f01-40b7-964f-796baee7620f admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:25.624992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ca4ef-7f01-40b7-964f-796baee7620f admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:25.629551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852f18c-8cf8-4efa-aaec-f31a63f95849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:25.630221 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852f18c-8cf8-4efa-aaec-f31a63f95849 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:25.630502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852f18c-8cf8-4efa-aaec-f31a63f95849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:25.630777 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852f18c-8cf8-4efa-aaec-f31a63f95849 admin admin] PATH_INFO: `/v3/projects/259d29ba6f444cc794cecf5a3386363c/users/bcc55de64a21428a87b8439d9eda7b28/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:25.647000 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852f18c-8cf8-4efa-aaec-f31a63f95849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=259d29ba6f444cc794cecf5a3386363c, user_id=bcc55de64a21428a87b8439d9eda7b28, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:25.649809 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852f18c-8cf8-4efa-aaec-f31a63f95849 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:25.662617 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 987/1990] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:47:25 2022] PUT /identity/v3/projects/259d29ba6f444cc794cecf5a3386363c/users/bcc55de64a21428a87b8439d9eda7b28/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:47:25.674710 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68094cf1-c076-4c58-8214-b635a0aeb1d4 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:25.674993 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68094cf1-c076-4c58-8214-b635a0aeb1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:25.675263 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68094cf1-c076-4c58-8214-b635a0aeb1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:25.702381 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-68094cf1-c076-4c58-8214-b635a0aeb1d4 None None] MFA Rules not processed for user `bcc55de64a21428a87b8439d9eda7b28`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:47:25.703591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-68094cf1-c076-4c58-8214-b635a0aeb1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:47:25.766845 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1004/1991] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:47:25 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:47:25.771320 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852f18c-8cf8-4efa-aaec-f31a63f95849 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:25.831495 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852f18c-8cf8-4efa-aaec-f31a63f95849 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:25.836059 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc65a5d-8e54-4c78-9343-4aaded1d0926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:25.836658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc65a5d-8e54-4c78-9343-4aaded1d0926 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:25.836938 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc65a5d-8e54-4c78-9343-4aaded1d0926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:25.837219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc65a5d-8e54-4c78-9343-4aaded1d0926 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:25.837839 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc65a5d-8e54-4c78-9343-4aaded1d0926 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:25.839757 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc65a5d-8e54-4c78-9343-4aaded1d0926 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:25.866090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 988/1992] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:47:25 2022] POST /identity/v3/projects => generated 403 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:47:25.870219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68094cf1-c076-4c58-8214-b635a0aeb1d4 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:25.880810 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68094cf1-c076-4c58-8214-b635a0aeb1d4 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:25.925160 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:47:25.937801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed736aa-d207-4749-9246-74ec5c2a2a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:25.938429 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed736aa-d207-4749-9246-74ec5c2a2a43 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:25.938711 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed736aa-d207-4749-9246-74ec5c2a2a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:25.938981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed736aa-d207-4749-9246-74ec5c2a2a43 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:25.939629 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed736aa-d207-4749-9246-74ec5c2a2a43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:25.941549 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed736aa-d207-4749-9246-74ec5c2a2a43 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:25.961743 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1005/1993] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:47:25 2022] POST /identity/v3/users => generated 308 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:47:25.965749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc65a5d-8e54-4c78-9343-4aaded1d0926 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:25.976421 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc65a5d-8e54-4c78-9343-4aaded1d0926 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:25.980955 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c2129-0ac6-4bce-a5a6-ae9a9126de15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:25.981542 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c2129-0ac6-4bce-a5a6-ae9a9126de15 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:25.981820 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c2129-0ac6-4bce-a5a6-ae9a9126de15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:25.982092 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c2129-0ac6-4bce-a5a6-ae9a9126de15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:25.982715 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c2129-0ac6-4bce-a5a6-ae9a9126de15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:25.984324 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c2129-0ac6-4bce-a5a6-ae9a9126de15 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:25.996206 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 989/1994] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:47:26.000159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed736aa-d207-4749-9246-74ec5c2a2a43 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:26.009950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed736aa-d207-4749-9246-74ec5c2a2a43 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:26.014474 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd68a0-70de-4164-b536-6901c25fa111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:26.015116 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd68a0-70de-4164-b536-6901c25fa111 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:26.015394 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd68a0-70de-4164-b536-6901c25fa111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:26.015945 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd68a0-70de-4164-b536-6901c25fa111 admin admin] PATH_INFO: `/v3/projects/8b559e5e1d9349698c2e06594a73ca68/users/e62e4681f7494a1b899db46937cc7776/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:26.030217 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd68a0-70de-4164-b536-6901c25fa111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b559e5e1d9349698c2e06594a73ca68, user_id=e62e4681f7494a1b899db46937cc7776, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:26.032888 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd68a0-70de-4164-b536-6901c25fa111 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:26.046411 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1006/1995] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:47:25 2022] PUT /identity/v3/projects/8b559e5e1d9349698c2e06594a73ca68/users/e62e4681f7494a1b899db46937cc7776/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:47:26.050870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c2129-0ac6-4bce-a5a6-ae9a9126de15 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:26.060795 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c2129-0ac6-4bce-a5a6-ae9a9126de15 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:26.115491 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa106dd-a1c1-410b-8274-9fd9847b63fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:26.116114 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa106dd-a1c1-410b-8274-9fd9847b63fc admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:26.116331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa106dd-a1c1-410b-8274-9fd9847b63fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:26.116604 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa106dd-a1c1-410b-8274-9fd9847b63fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:26.117232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa106dd-a1c1-410b-8274-9fd9847b63fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:26.118869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa106dd-a1c1-410b-8274-9fd9847b63fc admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:26.128454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 990/1996] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:47:26.132595 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd68a0-70de-4164-b536-6901c25fa111 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:47:26.142805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd68a0-70de-4164-b536-6901c25fa111 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:47:26.147358 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9221c0-12c4-45b4-ae89-efd404736290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:47:26.148040 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9221c0-12c4-45b4-ae89-efd404736290 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:26.148337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9221c0-12c4-45b4-ae89-efd404736290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:26.148591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9221c0-12c4-45b4-ae89-efd404736290 admin admin] PATH_INFO: `/v3/projects/8b559e5e1d9349698c2e06594a73ca68/users/e62e4681f7494a1b899db46937cc7776/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:26.150439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9221c0-12c4-45b4-ae89-efd404736290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b559e5e1d9349698c2e06594a73ca68, user_id=e62e4681f7494a1b899db46937cc7776, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:47:26.153101 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9221c0-12c4-45b4-ae89-efd404736290 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:47:26.163472 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1007/1997] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:47:26 2022] PUT /identity/v3/projects/8b559e5e1d9349698c2e06594a73ca68/users/e62e4681f7494a1b899db46937cc7776/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:47:26.173522 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1deed-c226-49c4-aeaa-c0bf670b5737 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:47:26.174371 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1deed-c226-49c4-aeaa-c0bf670b5737 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:47:26.174644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1deed-c226-49c4-aeaa-c0bf670b5737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:47:26.193918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:47:26.203502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-36d1deed-c226-49c4-aeaa-c0bf670b5737 None None] MFA Rules not processed for user `e62e4681f7494a1b899db46937cc7776`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:47:26.204732 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-36d1deed-c226-49c4-aeaa-c0bf670b5737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:47:26.272042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 991/1998] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:47:26 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:47:36.160042 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=22,keystone:INSERT=5 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:47:36.269098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=30 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:48:17.546146 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9221c0-12c4-45b4-ae89-efd404736290 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:48:17.547977 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-7e9221c0-12c4-45b4-ae89-efd404736290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:48:17.605016 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9221c0-12c4-45b4-ae89-efd404736290 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:48:17.608545 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b1c25-ec89-40ad-a65d-194d2ed874e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:48:17.609142 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b1c25-ec89-40ad-a65d-194d2ed874e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:48:17.609416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b1c25-ec89-40ad-a65d-194d2ed874e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:48:17.609681 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b1c25-ec89-40ad-a65d-194d2ed874e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:48:17.611350 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-778b1c25-ec89-40ad-a65d-194d2ed874e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:48:17.623193 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b1c25-ec89-40ad-a65d-194d2ed874e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:48:17.624653 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b1c25-ec89-40ad-a65d-194d2ed874e1 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:48:17.632525 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1008/1999] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:48:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:48:27.628907 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=26 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:49:16.443306 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1deed-c226-49c4-aeaa-c0bf670b5737 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:49:16.444700 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-36d1deed-c226-49c4-aeaa-c0bf670b5737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:49:16.511508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1deed-c226-49c4-aeaa-c0bf670b5737 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:49:16.515403 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29082e-6eec-4f85-8609-f3dd95bfcc50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:49:16.516038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29082e-6eec-4f85-8609-f3dd95bfcc50 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:49:16.516838 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29082e-6eec-4f85-8609-f3dd95bfcc50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:49:16.517110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29082e-6eec-4f85-8609-f3dd95bfcc50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:49:16.518779 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2a29082e-6eec-4f85-8609-f3dd95bfcc50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:49:16.588003 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29082e-6eec-4f85-8609-f3dd95bfcc50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:49:16.589409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29082e-6eec-4f85-8609-f3dd95bfcc50 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:49:16.598906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 992/2000] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:49:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:49:16.659975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b1c25-ec89-40ad-a65d-194d2ed874e1 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:49:16.668920 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b1c25-ec89-40ad-a65d-194d2ed874e1 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:49:16.672414 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f56fa-5d01-4575-8706-456dcda5a7d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:49:16.673027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f56fa-5d01-4575-8706-456dcda5a7d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:49:16.673302 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f56fa-5d01-4575-8706-456dcda5a7d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:49:16.673562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f56fa-5d01-4575-8706-456dcda5a7d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:49:16.675224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e73f56fa-5d01-4575-8706-456dcda5a7d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:49:16.744087 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f56fa-5d01-4575-8706-456dcda5a7d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:49:16.745530 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f56fa-5d01-4575-8706-456dcda5a7d4 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:49:16.755146 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1009/2001] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:49:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:49:16.837196 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29082e-6eec-4f85-8609-f3dd95bfcc50 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:49:16.848529 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29082e-6eec-4f85-8609-f3dd95bfcc50 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:49:16.852236 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d97050-27b7-4255-bae0-514b142b2165 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:49:16.853016 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d97050-27b7-4255-bae0-514b142b2165 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:49:16.853532 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d97050-27b7-4255-bae0-514b142b2165 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:49:16.853918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d97050-27b7-4255-bae0-514b142b2165 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:49:16.863399 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d97050-27b7-4255-bae0-514b142b2165 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:49:16.864904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d97050-27b7-4255-bae0-514b142b2165 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:49:16.874266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 993/2002] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:49:16 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:49:26.750327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=33 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:49:26.870022 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=64 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:51:17.009703 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f56fa-5d01-4575-8706-456dcda5a7d4 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:17.011070 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e73f56fa-5d01-4575-8706-456dcda5a7d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:51:17.046152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f56fa-5d01-4575-8706-456dcda5a7d4 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:17.066418 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdac75-775c-49c9-b5ef-160edd3db87c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:17.067402 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdac75-775c-49c9-b5ef-160edd3db87c service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:17.067402 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdac75-775c-49c9-b5ef-160edd3db87c service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:17.067972 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdac75-775c-49c9-b5ef-160edd3db87c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:17.069579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-babdac75-775c-49c9-b5ef-160edd3db87c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:51:17.149044 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdac75-775c-49c9-b5ef-160edd3db87c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:17.150475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdac75-775c-49c9-b5ef-160edd3db87c service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:17.159717 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1010/2003] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:51:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:51:17.352794 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d97050-27b7-4255-bae0-514b142b2165 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:17.356172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c5d97050-27b7-4255-bae0-514b142b2165 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:51:17.510539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d97050-27b7-4255-bae0-514b142b2165 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:17.517559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ce2e6-4b40-4373-8ea9-bafb06333f95 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:17.518163 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ce2e6-4b40-4373-8ea9-bafb06333f95 service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:17.518443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ce2e6-4b40-4373-8ea9-bafb06333f95 service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:17.519263 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ce2e6-4b40-4373-8ea9-bafb06333f95 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:17.520634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-261ce2e6-4b40-4373-8ea9-bafb06333f95 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:51:17.544769 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ce2e6-4b40-4373-8ea9-bafb06333f95 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:17.546557 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ce2e6-4b40-4373-8ea9-bafb06333f95 service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:17.565884 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 994/2004] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:51:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:51:17.668126 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdac75-775c-49c9-b5ef-160edd3db87c service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:17.739113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdac75-775c-49c9-b5ef-160edd3db87c service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:17.743974 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e696d-11e8-4707-9cea-159d0e227b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:17.744698 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e696d-11e8-4707-9cea-159d0e227b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:17.744978 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e696d-11e8-4707-9cea-159d0e227b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:17.745253 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e696d-11e8-4707-9cea-159d0e227b2a admin admin] PATH_INFO: `/v3/users/bcc55de64a21428a87b8439d9eda7b28` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:17.746943 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e696d-11e8-4707-9cea-159d0e227b2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcc55de64a21428a87b8439d9eda7b28)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:17.748923 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e696d-11e8-4707-9cea-159d0e227b2a admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:17.802200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-833e696d-11e8-4707-9cea-159d0e227b2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcc55de64a21428a87b8439d9eda7b28'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:17.814987 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-833e696d-11e8-4707-9cea-159d0e227b2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcc55de64a21428a87b8439d9eda7b28'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:17.817664 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:51:17.822035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-833e696d-11e8-4707-9cea-159d0e227b2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcc55de64a21428a87b8439d9eda7b28'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:17.823123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-833e696d-11e8-4707-9cea-159d0e227b2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcc55de64a21428a87b8439d9eda7b28'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:17.831502 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-833e696d-11e8-4707-9cea-159d0e227b2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcc55de64a21428a87b8439d9eda7b28'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:17.837688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1011/2005] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:51:17 2022] DELETE /identity/v3/users/bcc55de64a21428a87b8439d9eda7b28 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:51:17.842161 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f758302-7e1c-4de9-8861-82de47438b8f None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:17.842453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f758302-7e1c-4de9-8861-82de47438b8f None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:17.842732 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f758302-7e1c-4de9-8861-82de47438b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:17.861991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-0f758302-7e1c-4de9-8861-82de47438b8f None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:51:17.863190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-0f758302-7e1c-4de9-8861-82de47438b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:51:17.921536 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 995/2006] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:51:17.931340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e696d-11e8-4707-9cea-159d0e227b2a admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:17.932772 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-833e696d-11e8-4707-9cea-159d0e227b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:51:17.996671 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e696d-11e8-4707-9cea-159d0e227b2a admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:18.000252 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8d2ba-eae4-49da-ac8b-54c535beac42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:18.000847 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8d2ba-eae4-49da-ac8b-54c535beac42 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:18.001122 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8d2ba-eae4-49da-ac8b-54c535beac42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:18.001391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8d2ba-eae4-49da-ac8b-54c535beac42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:18.010097 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8d2ba-eae4-49da-ac8b-54c535beac42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:18.011450 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8d2ba-eae4-49da-ac8b-54c535beac42 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:18.020981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1012/2007] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:51:18.336179 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f758302-7e1c-4de9-8861-82de47438b8f None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:18.337750 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-0f758302-7e1c-4de9-8861-82de47438b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:51:18.354500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f758302-7e1c-4de9-8861-82de47438b8f None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:18.361407 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f225a96-d528-45fd-82da-22a34ec7513d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:18.362030 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f225a96-d528-45fd-82da-22a34ec7513d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:18.362453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f225a96-d528-45fd-82da-22a34ec7513d admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:18.362584 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f225a96-d528-45fd-82da-22a34ec7513d admin admin] PATH_INFO: `/v3/projects/259d29ba6f444cc794cecf5a3386363c` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:18.364120 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f225a96-d528-45fd-82da-22a34ec7513d admin admin] RBAC: Authorizing `identity:delete_project(project_id=259d29ba6f444cc794cecf5a3386363c)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:18.366028 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f225a96-d528-45fd-82da-22a34ec7513d admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:18.417563 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:51:18.421760 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3f225a96-d528-45fd-82da-22a34ec7513d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '259d29ba6f444cc794cecf5a3386363c', 'request_id': 'req-3f225a96-d528-45fd-82da-22a34ec7513d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:18.437058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3f225a96-d528-45fd-82da-22a34ec7513d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '259d29ba6f444cc794cecf5a3386363c', 'request_id': 'req-3f225a96-d528-45fd-82da-22a34ec7513d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:18.443951 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3f225a96-d528-45fd-82da-22a34ec7513d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '259d29ba6f444cc794cecf5a3386363c', 'request_id': 'req-3f225a96-d528-45fd-82da-22a34ec7513d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:18.456396 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3f225a96-d528-45fd-82da-22a34ec7513d admin admin] The token cache is being invalidate because project 259d29ba6f444cc794cecf5a3386363c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:51:18.456686 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3f225a96-d528-45fd-82da-22a34ec7513d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:18.460223 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3f225a96-d528-45fd-82da-22a34ec7513d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:18.462169 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 996/2008] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:51:18 2022] DELETE /identity/v3/projects/259d29ba6f444cc794cecf5a3386363c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:51:18.466354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8d2ba-eae4-49da-ac8b-54c535beac42 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:18.476107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8d2ba-eae4-49da-ac8b-54c535beac42 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:18.550972 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fcc27-dc87-4c74-b0ea-c91765dcea2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:18.552111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fcc27-dc87-4c74-b0ea-c91765dcea2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:18.552294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fcc27-dc87-4c74-b0ea-c91765dcea2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:18.552667 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fcc27-dc87-4c74-b0ea-c91765dcea2b admin admin] PATH_INFO: `/v3/users/e62e4681f7494a1b899db46937cc7776` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:18.553914 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fcc27-dc87-4c74-b0ea-c91765dcea2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e62e4681f7494a1b899db46937cc7776)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:18.559920 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fcc27-dc87-4c74-b0ea-c91765dcea2b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:18.620865 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-127fcc27-dc87-4c74-b0ea-c91765dcea2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e62e4681f7494a1b899db46937cc7776'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:18.637909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-127fcc27-dc87-4c74-b0ea-c91765dcea2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e62e4681f7494a1b899db46937cc7776'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:18.648678 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-127fcc27-dc87-4c74-b0ea-c91765dcea2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e62e4681f7494a1b899db46937cc7776'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:18.649325 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-127fcc27-dc87-4c74-b0ea-c91765dcea2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e62e4681f7494a1b899db46937cc7776'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:18.661407 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-127fcc27-dc87-4c74-b0ea-c91765dcea2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e62e4681f7494a1b899db46937cc7776'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:18.664518 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:51:18.673417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1013/2009] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:51:18 2022] DELETE /identity/v3/users/e62e4681f7494a1b899db46937cc7776 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:51:18.967113 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f225a96-d528-45fd-82da-22a34ec7513d admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:18.968918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3f225a96-d528-45fd-82da-22a34ec7513d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:51:18.969068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fcc27-dc87-4c74-b0ea-c91765dcea2b admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:18.974949 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-127fcc27-dc87-4c74-b0ea-c91765dcea2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:51:18.984126 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f225a96-d528-45fd-82da-22a34ec7513d admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:19.047874 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c9b46-320b-4066-911b-9c6cb776392b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:19.048407 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c9b46-320b-4066-911b-9c6cb776392b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:19.048687 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c9b46-320b-4066-911b-9c6cb776392b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:19.048959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c9b46-320b-4066-911b-9c6cb776392b admin admin] PATH_INFO: `/v3/projects/8b559e5e1d9349698c2e06594a73ca68` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:19.050103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c9b46-320b-4066-911b-9c6cb776392b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b559e5e1d9349698c2e06594a73ca68)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:19.052042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c9b46-320b-4066-911b-9c6cb776392b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:19.086850 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fcc27-dc87-4c74-b0ea-c91765dcea2b admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:19.099104 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ef5c9b46-320b-4066-911b-9c6cb776392b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b559e5e1d9349698c2e06594a73ca68', 'request_id': 'req-ef5c9b46-320b-4066-911b-9c6cb776392b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:19.106485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ef5c9b46-320b-4066-911b-9c6cb776392b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b559e5e1d9349698c2e06594a73ca68', 'request_id': 'req-ef5c9b46-320b-4066-911b-9c6cb776392b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:19.112588 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ef5c9b46-320b-4066-911b-9c6cb776392b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b559e5e1d9349698c2e06594a73ca68', 'request_id': 'req-ef5c9b46-320b-4066-911b-9c6cb776392b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:19.124411 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ef5c9b46-320b-4066-911b-9c6cb776392b admin admin] The token cache is being invalidate because project 8b559e5e1d9349698c2e06594a73ca68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:51:19.124698 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ef5c9b46-320b-4066-911b-9c6cb776392b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:19.125341 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ef5c9b46-320b-4066-911b-9c6cb776392b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:51:19.126830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 997/2010] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:51:18 2022] DELETE /identity/v3/projects/8b559e5e1d9349698c2e06594a73ca68 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:51:19.131546 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64a8b5-8f94-4fa8-8d30-5053effea7ef None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:19.131859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64a8b5-8f94-4fa8-8d30-5053effea7ef None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:19.132150 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64a8b5-8f94-4fa8-8d30-5053effea7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:19.152781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-6c64a8b5-8f94-4fa8-8d30-5053effea7ef None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:51:19.153007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ca74d-d389-4a48-9534-8f7d6ad3fc32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:19.153441 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ca74d-d389-4a48-9534-8f7d6ad3fc32 service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:19.153722 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ca74d-d389-4a48-9534-8f7d6ad3fc32 service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:19.153994 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ca74d-d389-4a48-9534-8f7d6ad3fc32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:19.154107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-6c64a8b5-8f94-4fa8-8d30-5053effea7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:51:19.155742 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-361ca74d-d389-4a48-9534-8f7d6ad3fc32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:51:19.167444 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ca74d-d389-4a48-9534-8f7d6ad3fc32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:19.169108 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ca74d-d389-4a48-9534-8f7d6ad3fc32 service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:19.179439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1014/2011] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:51:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:51:19.212002 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 998/2012] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:51:19.219914 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b02af2-1d6a-4337-bb5a-6a6469de6124 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:19.220205 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b02af2-1d6a-4337-bb5a-6a6469de6124 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:19.220483 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b02af2-1d6a-4337-bb5a-6a6469de6124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:19.239783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-e1b02af2-1d6a-4337-bb5a-6a6469de6124 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:51:19.240984 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e1b02af2-1d6a-4337-bb5a-6a6469de6124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:51:19.249258 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1015/2013] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:51:19.253541 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64a8b5-8f94-4fa8-8d30-5053effea7ef None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:19.259695 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:51:19.264417 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64a8b5-8f94-4fa8-8d30-5053effea7ef None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:19.270207 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0bfe5-f084-44fe-b041-1e8aa4d9881e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:19.270981 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0bfe5-f084-44fe-b041-1e8aa4d9881e admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:19.271430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0bfe5-f084-44fe-b041-1e8aa4d9881e admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:19.271802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0bfe5-f084-44fe-b041-1e8aa4d9881e admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:19.272605 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0bfe5-f084-44fe-b041-1e8aa4d9881e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:51:19.272907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0bfe5-f084-44fe-b041-1e8aa4d9881e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:19.274486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0bfe5-f084-44fe-b041-1e8aa4d9881e admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:19.285783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 999/2014] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:51:19.290035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b02af2-1d6a-4337-bb5a-6a6469de6124 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:19.299224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b02af2-1d6a-4337-bb5a-6a6469de6124 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:19.307105 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa6a25-063c-4a2d-bb5e-e06dae1e86ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:19.307757 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa6a25-063c-4a2d-bb5e-e06dae1e86ec admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:19.308039 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa6a25-063c-4a2d-bb5e-e06dae1e86ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:19.308312 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa6a25-063c-4a2d-bb5e-e06dae1e86ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:19.309508 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa6a25-063c-4a2d-bb5e-e06dae1e86ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:19.311438 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa6a25-063c-4a2d-bb5e-e06dae1e86ec admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:19.338256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1016/2015] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:51:19 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:51:19.342372 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0bfe5-f084-44fe-b041-1e8aa4d9881e admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:19.352554 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0bfe5-f084-44fe-b041-1e8aa4d9881e admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:19.407912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2aa118-89f3-46ea-b070-fcd7ee2bd2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:19.408546 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2aa118-89f3-46ea-b070-fcd7ee2bd2ad admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:19.408795 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2aa118-89f3-46ea-b070-fcd7ee2bd2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:19.409046 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2aa118-89f3-46ea-b070-fcd7ee2bd2ad admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:19.409661 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2aa118-89f3-46ea-b070-fcd7ee2bd2ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:19.411550 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2aa118-89f3-46ea-b070-fcd7ee2bd2ad admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:19.432550 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1000/2016] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:51:19 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:51:19.437177 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa6a25-063c-4a2d-bb5e-e06dae1e86ec admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:19.447253 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa6a25-063c-4a2d-bb5e-e06dae1e86ec admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:19.451826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adc38d-17ad-44b7-8334-8e7a6cf06b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:19.452429 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adc38d-17ad-44b7-8334-8e7a6cf06b1f admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:19.452710 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adc38d-17ad-44b7-8334-8e7a6cf06b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:19.452986 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adc38d-17ad-44b7-8334-8e7a6cf06b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:19.453621 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adc38d-17ad-44b7-8334-8e7a6cf06b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:19.455224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adc38d-17ad-44b7-8334-8e7a6cf06b1f admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:19.464925 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1017/2017] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:51:19.468577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2aa118-89f3-46ea-b070-fcd7ee2bd2ad admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:19.478361 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2aa118-89f3-46ea-b070-fcd7ee2bd2ad admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:19.483671 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe8eae-2e0a-4976-b32d-5825964dc467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:19.484269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe8eae-2e0a-4976-b32d-5825964dc467 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:19.484547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe8eae-2e0a-4976-b32d-5825964dc467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:19.484813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe8eae-2e0a-4976-b32d-5825964dc467 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:19.485439 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe8eae-2e0a-4976-b32d-5825964dc467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:19.487000 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe8eae-2e0a-4976-b32d-5825964dc467 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:19.498499 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1001/2018] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:51:19.502678 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adc38d-17ad-44b7-8334-8e7a6cf06b1f admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:19.512812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adc38d-17ad-44b7-8334-8e7a6cf06b1f admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:19.517680 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1f899-ec59-41cc-abf7-72eb09fe1c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:19.518330 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1f899-ec59-41cc-abf7-72eb09fe1c98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:19.518611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1f899-ec59-41cc-abf7-72eb09fe1c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:19.518888 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1f899-ec59-41cc-abf7-72eb09fe1c98 admin admin] PATH_INFO: `/v3/projects/69bb5ae23eb1436c915d0e3407012255/users/3cf3a49826ac480b9737068165545e0f/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:19.533822 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1f899-ec59-41cc-abf7-72eb09fe1c98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69bb5ae23eb1436c915d0e3407012255, user_id=3cf3a49826ac480b9737068165545e0f, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:19.536521 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1f899-ec59-41cc-abf7-72eb09fe1c98 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:19.549895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1018/2019] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:51:19 2022] PUT /identity/v3/projects/69bb5ae23eb1436c915d0e3407012255/users/3cf3a49826ac480b9737068165545e0f/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:51:19.554926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff647aa-5ceb-458b-8621-ad4dd97ec13a None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:19.555279 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff647aa-5ceb-458b-8621-ad4dd97ec13a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:19.555641 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff647aa-5ceb-458b-8621-ad4dd97ec13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:19.575217 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-eff647aa-5ceb-458b-8621-ad4dd97ec13a None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:51:19.576508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-eff647aa-5ceb-458b-8621-ad4dd97ec13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:51:19.644058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1002/2020] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:51:19.653535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1f899-ec59-41cc-abf7-72eb09fe1c98 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:19.655223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d9e1f899-ec59-41cc-abf7-72eb09fe1c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:51:19.660491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:51:19.713764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1f899-ec59-41cc-abf7-72eb09fe1c98 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:19.717353 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f87f83-907c-4808-a4cb-bf5baf80f799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:19.717950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f87f83-907c-4808-a4cb-bf5baf80f799 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:19.718223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f87f83-907c-4808-a4cb-bf5baf80f799 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:19.718489 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f87f83-907c-4808-a4cb-bf5baf80f799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:19.727700 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f87f83-907c-4808-a4cb-bf5baf80f799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:19.729154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f87f83-907c-4808-a4cb-bf5baf80f799 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:19.739085 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1019/2021] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:51:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:51:20.352144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65265d1d-ca6e-44d5-b273-94b9a94c214d None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:20.352553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65265d1d-ca6e-44d5-b273-94b9a94c214d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:20.352942 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65265d1d-ca6e-44d5-b273-94b9a94c214d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:20.380504 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-65265d1d-ca6e-44d5-b273-94b9a94c214d None None] MFA Rules not processed for user `3cf3a49826ac480b9737068165545e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:51:20.381633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-65265d1d-ca6e-44d5-b273-94b9a94c214d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:51:20.443983 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:51:20.452001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1003/2022] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:51:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:51:20.452830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f87f83-907c-4808-a4cb-bf5baf80f799 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:20.462254 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f87f83-907c-4808-a4cb-bf5baf80f799 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:20.467333 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a484f84-f336-4bb6-ab2d-fa326b6ae6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:20.467964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a484f84-f336-4bb6-ab2d-fa326b6ae6d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:20.468240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a484f84-f336-4bb6-ab2d-fa326b6ae6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:20.468516 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a484f84-f336-4bb6-ab2d-fa326b6ae6d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:20.469142 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a484f84-f336-4bb6-ab2d-fa326b6ae6d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:20.471039 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a484f84-f336-4bb6-ab2d-fa326b6ae6d5 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:20.498609 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1020/2023] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:51:20 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:51:20.502745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65265d1d-ca6e-44d5-b273-94b9a94c214d None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:20.514396 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65265d1d-ca6e-44d5-b273-94b9a94c214d None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:20.575381 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eac67c-9449-4015-9189-af9bfa8e72a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:20.576037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eac67c-9449-4015-9189-af9bfa8e72a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:20.576317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eac67c-9449-4015-9189-af9bfa8e72a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:20.576590 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eac67c-9449-4015-9189-af9bfa8e72a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:20.577229 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eac67c-9449-4015-9189-af9bfa8e72a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:20.579129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eac67c-9449-4015-9189-af9bfa8e72a6 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:20.604474 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1004/2024] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:51:20 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:51:20.605625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a484f84-f336-4bb6-ab2d-fa326b6ae6d5 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:20.615964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a484f84-f336-4bb6-ab2d-fa326b6ae6d5 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:20.620674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5fc6b-d8f5-4655-85ee-c1df9125e0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:20.621820 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5fc6b-d8f5-4655-85ee-c1df9125e0d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:20.622100 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5fc6b-d8f5-4655-85ee-c1df9125e0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:20.622370 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5fc6b-d8f5-4655-85ee-c1df9125e0d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:20.622997 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5fc6b-d8f5-4655-85ee-c1df9125e0d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:20.624611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5fc6b-d8f5-4655-85ee-c1df9125e0d5 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:20.635524 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1021/2025] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:51:20.640342 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eac67c-9449-4015-9189-af9bfa8e72a6 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:20.658434 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eac67c-9449-4015-9189-af9bfa8e72a6 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:20.664371 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bf3ac-6388-481f-9346-962e958aca8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:20.665153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bf3ac-6388-481f-9346-962e958aca8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:20.665870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bf3ac-6388-481f-9346-962e958aca8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:20.666348 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bf3ac-6388-481f-9346-962e958aca8c admin admin] PATH_INFO: `/v3/projects/c7c2f47dede54bf091d9e875ba974b4b/users/98fc5376ab1444a597fbfc5c2da8ed37/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:20.682638 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bf3ac-6388-481f-9346-962e958aca8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7c2f47dede54bf091d9e875ba974b4b, user_id=98fc5376ab1444a597fbfc5c2da8ed37, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:20.686214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bf3ac-6388-481f-9346-962e958aca8c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:20.698702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1005/2026] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:51:20 2022] PUT /identity/v3/projects/c7c2f47dede54bf091d9e875ba974b4b/users/98fc5376ab1444a597fbfc5c2da8ed37/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:51:20.702665 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5fc6b-d8f5-4655-85ee-c1df9125e0d5 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:20.712338 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5fc6b-d8f5-4655-85ee-c1df9125e0d5 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:20.772376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f78c44-0837-4537-8b1b-e2d04e19082d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:20.772975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f78c44-0837-4537-8b1b-e2d04e19082d admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:20.773249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f78c44-0837-4537-8b1b-e2d04e19082d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:20.773516 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f78c44-0837-4537-8b1b-e2d04e19082d admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:20.774136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f78c44-0837-4537-8b1b-e2d04e19082d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:20.775749 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f78c44-0837-4537-8b1b-e2d04e19082d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:20.785550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1022/2027] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:51:20.789690 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bf3ac-6388-481f-9346-962e958aca8c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:20.800539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bf3ac-6388-481f-9346-962e958aca8c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:20.805083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5e603-8140-4bcf-b6e5-24aa8a90da4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:20.805731 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5e603-8140-4bcf-b6e5-24aa8a90da4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:20.806034 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5e603-8140-4bcf-b6e5-24aa8a90da4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:20.806307 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5e603-8140-4bcf-b6e5-24aa8a90da4c admin admin] PATH_INFO: `/v3/projects/c7c2f47dede54bf091d9e875ba974b4b/users/98fc5376ab1444a597fbfc5c2da8ed37/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:20.808267 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5e603-8140-4bcf-b6e5-24aa8a90da4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7c2f47dede54bf091d9e875ba974b4b, user_id=98fc5376ab1444a597fbfc5c2da8ed37, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:20.810893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5e603-8140-4bcf-b6e5-24aa8a90da4c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:20.821152 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1006/2028] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:51:20 2022] PUT /identity/v3/projects/c7c2f47dede54bf091d9e875ba974b4b/users/98fc5376ab1444a597fbfc5c2da8ed37/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:51:21.441432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5c0a8-2eca-462f-aefe-c42f3091ead6 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:21.441714 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5c0a8-2eca-462f-aefe-c42f3091ead6 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:21.442015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5c0a8-2eca-462f-aefe-c42f3091ead6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:21.469614 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-3fe5c0a8-2eca-462f-aefe-c42f3091ead6 None None] MFA Rules not processed for user `98fc5376ab1444a597fbfc5c2da8ed37`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:51:21.470810 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-3fe5c0a8-2eca-462f-aefe-c42f3091ead6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:51:21.531327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:51:21.543061 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1023/2029] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:51:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:51:21.554369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5e603-8140-4bcf-b6e5-24aa8a90da4c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:51:21.556096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-7ce5e603-8140-4bcf-b6e5-24aa8a90da4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:51:21.617479 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5e603-8140-4bcf-b6e5-24aa8a90da4c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:51:21.625000 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8259a-7932-4c3a-a26c-e8e553fe4bfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:51:21.625810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8259a-7932-4c3a-a26c-e8e553fe4bfb service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:51:21.626674 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8259a-7932-4c3a-a26c-e8e553fe4bfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:51:21.626943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8259a-7932-4c3a-a26c-e8e553fe4bfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:51:21.685054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8259a-7932-4c3a-a26c-e8e553fe4bfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:51:21.686387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8259a-7932-4c3a-a26c-e8e553fe4bfb service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:51:21.695745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1007/2030] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:51:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:51:31.539954 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:51:31.692151 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=92,keystone:INSERT=5 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:52:15.866413 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5c0a8-2eca-462f-aefe-c42f3091ead6 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:52:15.877093 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5c0a8-2eca-462f-aefe-c42f3091ead6 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:52:15.880664 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e324b73-0875-43d5-b792-3a6ec6a8351f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:52:15.881274 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e324b73-0875-43d5-b792-3a6ec6a8351f service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:52:15.881556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e324b73-0875-43d5-b792-3a6ec6a8351f service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:52:15.881826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e324b73-0875-43d5-b792-3a6ec6a8351f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:52:15.883526 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-4e324b73-0875-43d5-b792-3a6ec6a8351f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:52:15.962491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e324b73-0875-43d5-b792-3a6ec6a8351f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:52:15.963961 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e324b73-0875-43d5-b792-3a6ec6a8351f service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:52:15.982748 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1024/2031] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:52:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:52:25.980154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=40 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:53:17.041542 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8259a-7932-4c3a-a26c-e8e553fe4bfb service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:17.042953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9ce8259a-7932-4c3a-a26c-e8e553fe4bfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:17.122772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8259a-7932-4c3a-a26c-e8e553fe4bfb service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:17.128663 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d9544-7311-4dab-9eee-20dd449c9719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:17.129519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d9544-7311-4dab-9eee-20dd449c9719 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:17.130005 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d9544-7311-4dab-9eee-20dd449c9719 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:17.130285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d9544-7311-4dab-9eee-20dd449c9719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:17.132091 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-428d9544-7311-4dab-9eee-20dd449c9719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:17.251030 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d9544-7311-4dab-9eee-20dd449c9719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:17.253073 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d9544-7311-4dab-9eee-20dd449c9719 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:17.274843 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1008/2032] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:53:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:53:17.282455 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e324b73-0875-43d5-b792-3a6ec6a8351f service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:17.291526 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e324b73-0875-43d5-b792-3a6ec6a8351f service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:17.295187 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c391bc-b231-432a-95c1-73f044a62032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:17.295820 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c391bc-b231-432a-95c1-73f044a62032 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:17.296100 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c391bc-b231-432a-95c1-73f044a62032 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:17.296366 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c391bc-b231-432a-95c1-73f044a62032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:17.298056 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-78c391bc-b231-432a-95c1-73f044a62032 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:17.382986 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c391bc-b231-432a-95c1-73f044a62032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:17.385041 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c391bc-b231-432a-95c1-73f044a62032 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:17.396565 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1025/2033] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:53:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:53:18.595183 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d9544-7311-4dab-9eee-20dd449c9719 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:18.604591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d9544-7311-4dab-9eee-20dd449c9719 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:18.608644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4cd21-4e0d-4fea-81f3-018c136a46a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:18.609281 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4cd21-4e0d-4fea-81f3-018c136a46a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:18.609564 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4cd21-4e0d-4fea-81f3-018c136a46a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:18.609836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4cd21-4e0d-4fea-81f3-018c136a46a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:18.612279 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-04a4cd21-4e0d-4fea-81f3-018c136a46a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:18.627656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4cd21-4e0d-4fea-81f3-018c136a46a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:18.629210 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4cd21-4e0d-4fea-81f3-018c136a46a3 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:18.643007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1009/2034] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:53:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:53:21.588227 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c391bc-b231-432a-95c1-73f044a62032 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:21.590493 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-78c391bc-b231-432a-95c1-73f044a62032 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:21.691751 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c391bc-b231-432a-95c1-73f044a62032 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:21.703422 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3782e5-aac9-4432-8207-f34d599bc89f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:21.704174 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3782e5-aac9-4432-8207-f34d599bc89f service placement] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:21.704499 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3782e5-aac9-4432-8207-f34d599bc89f service placement] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:21.707770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3782e5-aac9-4432-8207-f34d599bc89f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:21.708941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4cd21-4e0d-4fea-81f3-018c136a46a3 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:21.709531 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0d3782e5-aac9-4432-8207-f34d599bc89f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:21.710490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-04a4cd21-4e0d-4fea-81f3-018c136a46a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:21.727691 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3782e5-aac9-4432-8207-f34d599bc89f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:21.730601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3782e5-aac9-4432-8207-f34d599bc89f service placement] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:21.733760 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4cd21-4e0d-4fea-81f3-018c136a46a3 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:21.743711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b91bb4-3a44-461a-85b7-25b5b4e4ffe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:21.744465 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b91bb4-3a44-461a-85b7-25b5b4e4ffe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:21.745480 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b91bb4-3a44-461a-85b7-25b5b4e4ffe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:21.745828 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b91bb4-3a44-461a-85b7-25b5b4e4ffe2 admin admin] PATH_INFO: `/v3/users/dc5388b90d0847739330a095f4d5f106` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:21.748688 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b91bb4-3a44-461a-85b7-25b5b4e4ffe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc5388b90d0847739330a095f4d5f106)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:21.752194 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b91bb4-3a44-461a-85b7-25b5b4e4ffe2 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:21.755618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1026/2035] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:53:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:53:21.819508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-58b91bb4-3a44-461a-85b7-25b5b4e4ffe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc5388b90d0847739330a095f4d5f106'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:21.821181 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-58b91bb4-3a44-461a-85b7-25b5b4e4ffe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc5388b90d0847739330a095f4d5f106'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:21.839973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:53:21.841446 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-58b91bb4-3a44-461a-85b7-25b5b4e4ffe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc5388b90d0847739330a095f4d5f106'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:21.848991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-58b91bb4-3a44-461a-85b7-25b5b4e4ffe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc5388b90d0847739330a095f4d5f106'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:21.849762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-58b91bb4-3a44-461a-85b7-25b5b4e4ffe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc5388b90d0847739330a095f4d5f106'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:21.863357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1010/2036] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:53:21 2022] DELETE /identity/v3/users/dc5388b90d0847739330a095f4d5f106 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:53:22.080408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3782e5-aac9-4432-8207-f34d599bc89f service placement] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:22.107000 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3782e5-aac9-4432-8207-f34d599bc89f service placement] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:22.173765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:53:22.193564 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c053d-1b4f-499a-a870-9d16b4dc2615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:22.195493 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c053d-1b4f-499a-a870-9d16b4dc2615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:22.195493 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c053d-1b4f-499a-a870-9d16b4dc2615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:22.195493 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c053d-1b4f-499a-a870-9d16b4dc2615 admin admin] PATH_INFO: `/v3/projects/8122be51915c4754b86202ec79b4d197` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:22.206291 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c053d-1b4f-499a-a870-9d16b4dc2615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8122be51915c4754b86202ec79b4d197)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:22.208267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c053d-1b4f-499a-a870-9d16b4dc2615 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:22.261007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-615c053d-1b4f-499a-a870-9d16b4dc2615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8122be51915c4754b86202ec79b4d197', 'request_id': 'req-615c053d-1b4f-499a-a870-9d16b4dc2615'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:22.269408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-615c053d-1b4f-499a-a870-9d16b4dc2615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8122be51915c4754b86202ec79b4d197', 'request_id': 'req-615c053d-1b4f-499a-a870-9d16b4dc2615'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:22.277766 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-615c053d-1b4f-499a-a870-9d16b4dc2615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8122be51915c4754b86202ec79b4d197', 'request_id': 'req-615c053d-1b4f-499a-a870-9d16b4dc2615'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:22.289042 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-615c053d-1b4f-499a-a870-9d16b4dc2615 admin admin] The token cache is being invalidate because project 8122be51915c4754b86202ec79b4d197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:53:22.289915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-615c053d-1b4f-499a-a870-9d16b4dc2615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:22.290568 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-615c053d-1b4f-499a-a870-9d16b4dc2615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:22.292107 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1027/2037] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:53:22 2022] DELETE /identity/v3/projects/8122be51915c4754b86202ec79b4d197 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:53:22.297794 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb85092-161e-4f14-94f1-25ac616ebc63 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:22.298083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb85092-161e-4f14-94f1-25ac616ebc63 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:22.298401 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb85092-161e-4f14-94f1-25ac616ebc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:22.318918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-3cb85092-161e-4f14-94f1-25ac616ebc63 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:53:22.320291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3cb85092-161e-4f14-94f1-25ac616ebc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:22.384543 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1011/2038] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:53:22.395038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3445d72-0531-4d1a-92d9-67e8711feb8c None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:22.395376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3445d72-0531-4d1a-92d9-67e8711feb8c None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:22.395692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3445d72-0531-4d1a-92d9-67e8711feb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:22.423422 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-b3445d72-0531-4d1a-92d9-67e8711feb8c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:53:22.424738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b3445d72-0531-4d1a-92d9-67e8711feb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:22.434181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1028/2039] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:53:22.438964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb85092-161e-4f14-94f1-25ac616ebc63 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:22.450327 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb85092-161e-4f14-94f1-25ac616ebc63 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:22.465435 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4962c34-6cfa-4f13-945e-2a7cae5e6e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:22.466055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4962c34-6cfa-4f13-945e-2a7cae5e6e20 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:22.466335 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4962c34-6cfa-4f13-945e-2a7cae5e6e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:22.466606 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4962c34-6cfa-4f13-945e-2a7cae5e6e20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:22.467282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4962c34-6cfa-4f13-945e-2a7cae5e6e20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:53:22.467597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4962c34-6cfa-4f13-945e-2a7cae5e6e20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:22.469237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4962c34-6cfa-4f13-945e-2a7cae5e6e20 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:22.481986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1012/2040] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:53:22.486753 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3445d72-0531-4d1a-92d9-67e8711feb8c None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:22.498186 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3445d72-0531-4d1a-92d9-67e8711feb8c None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:22.503832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc031d-6eec-430d-9295-81a4c955516c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:22.504442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc031d-6eec-430d-9295-81a4c955516c admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:22.505132 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc031d-6eec-430d-9295-81a4c955516c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:22.506221 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc031d-6eec-430d-9295-81a4c955516c admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:22.506616 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc031d-6eec-430d-9295-81a4c955516c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:22.509226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc031d-6eec-430d-9295-81a4c955516c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:22.549553 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1029/2041] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:53:22 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:53:22.550538 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4962c34-6cfa-4f13-945e-2a7cae5e6e20 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:22.568873 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4962c34-6cfa-4f13-945e-2a7cae5e6e20 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:22.656271 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c41d87-ba45-45bb-8f5e-c1481debcdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:22.656892 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c41d87-ba45-45bb-8f5e-c1481debcdb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:22.657174 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c41d87-ba45-45bb-8f5e-c1481debcdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:22.657446 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c41d87-ba45-45bb-8f5e-c1481debcdb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:22.658077 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c41d87-ba45-45bb-8f5e-c1481debcdb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:22.660026 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c41d87-ba45-45bb-8f5e-c1481debcdb8 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:22.685343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1013/2042] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:53:22 2022] POST /identity/v3/users => generated 288 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:53:22.689013 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc031d-6eec-430d-9295-81a4c955516c admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:22.701986 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc031d-6eec-430d-9295-81a4c955516c admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:22.707663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb472f3b-8785-436d-b04c-15c24bf14af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:22.708136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb472f3b-8785-436d-b04c-15c24bf14af7 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:22.708496 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb472f3b-8785-436d-b04c-15c24bf14af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:22.708670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb472f3b-8785-436d-b04c-15c24bf14af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:22.709429 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb472f3b-8785-436d-b04c-15c24bf14af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:22.711057 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb472f3b-8785-436d-b04c-15c24bf14af7 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:22.722750 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1030/2043] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:53:22.726974 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c41d87-ba45-45bb-8f5e-c1481debcdb8 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:22.742802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c41d87-ba45-45bb-8f5e-c1481debcdb8 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:22.748099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5249109-bd15-4b27-8537-23de51e497f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:22.749194 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5249109-bd15-4b27-8537-23de51e497f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:22.749654 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5249109-bd15-4b27-8537-23de51e497f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:22.750120 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5249109-bd15-4b27-8537-23de51e497f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:22.751896 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5249109-bd15-4b27-8537-23de51e497f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:22.753690 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5249109-bd15-4b27-8537-23de51e497f8 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:22.766029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1014/2044] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:53:22.770181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb472f3b-8785-436d-b04c-15c24bf14af7 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:22.780080 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb472f3b-8785-436d-b04c-15c24bf14af7 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:22.784548 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04adf9-a50c-4d8a-9428-849f0c17f79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:22.785191 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04adf9-a50c-4d8a-9428-849f0c17f79e admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:22.785461 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04adf9-a50c-4d8a-9428-849f0c17f79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:22.785726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04adf9-a50c-4d8a-9428-849f0c17f79e admin admin] PATH_INFO: `/v3/projects/7023db393f774c0391e3bf29f6848513/users/576f7f3266d54240a072d9ae4edb4f57/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:22.803119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04adf9-a50c-4d8a-9428-849f0c17f79e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7023db393f774c0391e3bf29f6848513, user_id=576f7f3266d54240a072d9ae4edb4f57, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:22.806528 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04adf9-a50c-4d8a-9428-849f0c17f79e admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:22.832220 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1031/2045] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:53:22 2022] PUT /identity/v3/projects/7023db393f774c0391e3bf29f6848513/users/576f7f3266d54240a072d9ae4edb4f57/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:53:22.833892 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b91055-78e1-4da0-83bb-1f02b8e9b4d4 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:22.834127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b91055-78e1-4da0-83bb-1f02b8e9b4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:22.834404 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b91055-78e1-4da0-83bb-1f02b8e9b4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:22.869137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-e4b91055-78e1-4da0-83bb-1f02b8e9b4d4 None None] MFA Rules not processed for user `576f7f3266d54240a072d9ae4edb4f57`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:53:22.870920 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-e4b91055-78e1-4da0-83bb-1f02b8e9b4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:22.915992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:53:22.949347 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1015/2046] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:53:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:53:22.960817 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04adf9-a50c-4d8a-9428-849f0c17f79e admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:22.962667 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9e04adf9-a50c-4d8a-9428-849f0c17f79e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:23.038126 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04adf9-a50c-4d8a-9428-849f0c17f79e admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:23.043665 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c45016-cde8-4ce8-8cc3-660a5287fa94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:23.044387 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c45016-cde8-4ce8-8cc3-660a5287fa94 service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:23.044664 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c45016-cde8-4ce8-8cc3-660a5287fa94 service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:23.044934 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c45016-cde8-4ce8-8cc3-660a5287fa94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:23.051229 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:53:23.053359 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c45016-cde8-4ce8-8cc3-660a5287fa94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:23.056152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c45016-cde8-4ce8-8cc3-660a5287fa94 service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:23.067118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1032/2047] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:53:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:53:23.289855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b91055-78e1-4da0-83bb-1f02b8e9b4d4 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:23.354160 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b91055-78e1-4da0-83bb-1f02b8e9b4d4 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:23.358471 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675aa84-8590-4c4b-8f3e-800089f296b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:23.359126 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675aa84-8590-4c4b-8f3e-800089f296b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:23.359401 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675aa84-8590-4c4b-8f3e-800089f296b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:23.359731 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675aa84-8590-4c4b-8f3e-800089f296b3 admin admin] PATH_INFO: `/v3/users/576f7f3266d54240a072d9ae4edb4f57` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:23.361043 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675aa84-8590-4c4b-8f3e-800089f296b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=576f7f3266d54240a072d9ae4edb4f57)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:23.362939 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675aa84-8590-4c4b-8f3e-800089f296b3 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:23.408969 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-4675aa84-8590-4c4b-8f3e-800089f296b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '576f7f3266d54240a072d9ae4edb4f57'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:23.409678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-4675aa84-8590-4c4b-8f3e-800089f296b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '576f7f3266d54240a072d9ae4edb4f57'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:23.422479 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-4675aa84-8590-4c4b-8f3e-800089f296b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '576f7f3266d54240a072d9ae4edb4f57'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:23.431014 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-4675aa84-8590-4c4b-8f3e-800089f296b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '576f7f3266d54240a072d9ae4edb4f57'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:23.431857 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-4675aa84-8590-4c4b-8f3e-800089f296b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '576f7f3266d54240a072d9ae4edb4f57'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:23.444985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1016/2048] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:53:23 2022] DELETE /identity/v3/users/576f7f3266d54240a072d9ae4edb4f57 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:53:23.449631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28e650-53c4-4eeb-8b96-444739d6e69e None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:23.449984 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28e650-53c4-4eeb-8b96-444739d6e69e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:23.450374 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28e650-53c4-4eeb-8b96-444739d6e69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:23.469914 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-4e28e650-53c4-4eeb-8b96-444739d6e69e None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:53:23.471206 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-4e28e650-53c4-4eeb-8b96-444739d6e69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:23.538995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1033/2049] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:53:23.549516 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675aa84-8590-4c4b-8f3e-800089f296b3 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:23.550960 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-4675aa84-8590-4c4b-8f3e-800089f296b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:23.554739 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b0762-812f-4b9d-ba36-f7da7bcd02c5 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:23.555058 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b0762-812f-4b9d-ba36-f7da7bcd02c5 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:23.555332 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b0762-812f-4b9d-ba36-f7da7bcd02c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:23.574854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-208b0762-812f-4b9d-ba36-f7da7bcd02c5 None None] MFA Rules not processed for user `00374760bfa648e48dedae8aadb61c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:53:23.576113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-208b0762-812f-4b9d-ba36-f7da7bcd02c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:23.620360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675aa84-8590-4c4b-8f3e-800089f296b3 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:23.623909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32537b2-227a-407a-93b4-9b9ce546d52f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:23.624524 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32537b2-227a-407a-93b4-9b9ce546d52f service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:23.624800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32537b2-227a-407a-93b4-9b9ce546d52f service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:23.625076 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32537b2-227a-407a-93b4-9b9ce546d52f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:23.634264 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32537b2-227a-407a-93b4-9b9ce546d52f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:23.635699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32537b2-227a-407a-93b4-9b9ce546d52f service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:23.639928 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:53:23.641337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1034/2050] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:53:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:53:23.649680 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1017/2051] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:53:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:53:24.025066 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b0762-812f-4b9d-ba36-f7da7bcd02c5 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:24.033383 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32537b2-227a-407a-93b4-9b9ce546d52f service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:24.035278 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c32537b2-227a-407a-93b4-9b9ce546d52f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:24.035821 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b0762-812f-4b9d-ba36-f7da7bcd02c5 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:24.039891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2641a-1631-4acd-acfd-a266285f8aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:24.040752 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2641a-1631-4acd-acfd-a266285f8aab service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:24.041235 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2641a-1631-4acd-acfd-a266285f8aab service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:24.041720 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2641a-1631-4acd-acfd-a266285f8aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:24.044393 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-06f2641a-1631-4acd-acfd-a266285f8aab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:24.050701 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32537b2-227a-407a-93b4-9b9ce546d52f service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:24.055598 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2641a-1631-4acd-acfd-a266285f8aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:24.057333 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5346f2f-941d-4153-b3c1-b246ff6ed420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:24.057970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2641a-1631-4acd-acfd-a266285f8aab service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:24.058121 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5346f2f-941d-4153-b3c1-b246ff6ed420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:24.058404 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5346f2f-941d-4153-b3c1-b246ff6ed420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:24.058863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5346f2f-941d-4153-b3c1-b246ff6ed420 admin admin] PATH_INFO: `/v3/projects/7023db393f774c0391e3bf29f6848513` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:24.060062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5346f2f-941d-4153-b3c1-b246ff6ed420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7023db393f774c0391e3bf29f6848513)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:24.062173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5346f2f-941d-4153-b3c1-b246ff6ed420 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:24.068320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1035/2052] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:53:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:53:24.112418 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b5346f2f-941d-4153-b3c1-b246ff6ed420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7023db393f774c0391e3bf29f6848513', 'request_id': 'req-b5346f2f-941d-4153-b3c1-b246ff6ed420'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:24.121447 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b5346f2f-941d-4153-b3c1-b246ff6ed420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7023db393f774c0391e3bf29f6848513', 'request_id': 'req-b5346f2f-941d-4153-b3c1-b246ff6ed420'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:24.127370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b5346f2f-941d-4153-b3c1-b246ff6ed420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7023db393f774c0391e3bf29f6848513', 'request_id': 'req-b5346f2f-941d-4153-b3c1-b246ff6ed420'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:24.140495 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b5346f2f-941d-4153-b3c1-b246ff6ed420 admin admin] The token cache is being invalidate because project 7023db393f774c0391e3bf29f6848513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:53:24.140941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b5346f2f-941d-4153-b3c1-b246ff6ed420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:24.141585 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b5346f2f-941d-4153-b3c1-b246ff6ed420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:53:24.143164 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1018/2053] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:53:24 2022] DELETE /identity/v3/projects/7023db393f774c0391e3bf29f6848513 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:53:24.148413 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831c699-cb63-4dcc-bc7d-966467bb11a7 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:24.149245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831c699-cb63-4dcc-bc7d-966467bb11a7 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:24.149522 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831c699-cb63-4dcc-bc7d-966467bb11a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:24.169636 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-e831c699-cb63-4dcc-bc7d-966467bb11a7 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:53:24.170839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e831c699-cb63-4dcc-bc7d-966467bb11a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:24.230081 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1036/2054] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:53:24.238238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853cf75-5b51-4208-96a6-46e9586c5fbe None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:24.238528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853cf75-5b51-4208-96a6-46e9586c5fbe None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:24.238808 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853cf75-5b51-4208-96a6-46e9586c5fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:24.259500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-1853cf75-5b51-4208-96a6-46e9586c5fbe None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:53:24.260727 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1853cf75-5b51-4208-96a6-46e9586c5fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:24.268849 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1019/2055] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:53:24.273051 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831c699-cb63-4dcc-bc7d-966467bb11a7 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:24.282798 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831c699-cb63-4dcc-bc7d-966467bb11a7 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:24.287532 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368baaa-bda7-4feb-8b61-7394815fc59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:24.288160 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368baaa-bda7-4feb-8b61-7394815fc59a admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:24.288441 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368baaa-bda7-4feb-8b61-7394815fc59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:24.288715 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368baaa-bda7-4feb-8b61-7394815fc59a admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:24.289399 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f368baaa-bda7-4feb-8b61-7394815fc59a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:53:24.289686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f368baaa-bda7-4feb-8b61-7394815fc59a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:24.291291 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f368baaa-bda7-4feb-8b61-7394815fc59a admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:24.302998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1037/2056] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:53:24.307854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853cf75-5b51-4208-96a6-46e9586c5fbe None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:24.317456 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853cf75-5b51-4208-96a6-46e9586c5fbe None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:24.322768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c182163-1dea-4406-b7a1-73b93e019cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:24.323457 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c182163-1dea-4406-b7a1-73b93e019cca admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:24.323778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c182163-1dea-4406-b7a1-73b93e019cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:24.324075 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c182163-1dea-4406-b7a1-73b93e019cca admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:24.324708 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c182163-1dea-4406-b7a1-73b93e019cca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:24.326657 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c182163-1dea-4406-b7a1-73b93e019cca admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:24.353911 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1020/2057] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:53:24 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:53:24.358180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368baaa-bda7-4feb-8b61-7394815fc59a admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:24.369552 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368baaa-bda7-4feb-8b61-7394815fc59a admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:24.383800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:53:24.399891 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c182163-1dea-4406-b7a1-73b93e019cca admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:24.401525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-0c182163-1dea-4406-b7a1-73b93e019cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:24.443624 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469f654-4256-45c8-8394-fd8a2710707e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:24.444581 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469f654-4256-45c8-8394-fd8a2710707e admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:24.444867 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469f654-4256-45c8-8394-fd8a2710707e admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:24.445140 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469f654-4256-45c8-8394-fd8a2710707e admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:24.446354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469f654-4256-45c8-8394-fd8a2710707e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:24.448466 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469f654-4256-45c8-8394-fd8a2710707e admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:24.470792 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1038/2058] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:53:24 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:53:24.474705 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469f654-4256-45c8-8394-fd8a2710707e admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:24.484171 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469f654-4256-45c8-8394-fd8a2710707e admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:24.488708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a9f1c-13a0-450d-8754-9df8827cb94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:24.488954 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c182163-1dea-4406-b7a1-73b93e019cca admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:24.489288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a9f1c-13a0-450d-8754-9df8827cb94b admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:24.489572 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a9f1c-13a0-450d-8754-9df8827cb94b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:24.489843 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a9f1c-13a0-450d-8754-9df8827cb94b admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:24.490467 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a9f1c-13a0-450d-8754-9df8827cb94b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:24.492079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a9f1c-13a0-450d-8754-9df8827cb94b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:24.495124 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad12c6c-120c-476e-bfcc-0e92ddbd978e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:24.496237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad12c6c-120c-476e-bfcc-0e92ddbd978e service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:24.496237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad12c6c-120c-476e-bfcc-0e92ddbd978e service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:24.496481 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad12c6c-120c-476e-bfcc-0e92ddbd978e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:24.498985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9ad12c6c-120c-476e-bfcc-0e92ddbd978e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:24.505455 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1039/2059] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:53:24.510009 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a9f1c-13a0-450d-8754-9df8827cb94b admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:24.512508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad12c6c-120c-476e-bfcc-0e92ddbd978e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:24.514083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad12c6c-120c-476e-bfcc-0e92ddbd978e service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:24.520676 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a9f1c-13a0-450d-8754-9df8827cb94b admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:24.524061 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1021/2060] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:53:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:53:24.525540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8fb38-a447-444e-b6ae-bbaf508b540d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:24.526128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8fb38-a447-444e-b6ae-bbaf508b540d admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:24.526403 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8fb38-a447-444e-b6ae-bbaf508b540d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:24.526743 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8fb38-a447-444e-b6ae-bbaf508b540d admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:24.531859 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8fb38-a447-444e-b6ae-bbaf508b540d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:24.533743 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8fb38-a447-444e-b6ae-bbaf508b540d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:24.544843 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1040/2061] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:53:24.548846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad12c6c-120c-476e-bfcc-0e92ddbd978e service placement] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:24.559635 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad12c6c-120c-476e-bfcc-0e92ddbd978e service placement] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:24.564113 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae83c2-bbeb-4e9a-bc70-6d662358cf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:24.564758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae83c2-bbeb-4e9a-bc70-6d662358cf49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:24.565036 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae83c2-bbeb-4e9a-bc70-6d662358cf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:24.565307 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae83c2-bbeb-4e9a-bc70-6d662358cf49 admin admin] PATH_INFO: `/v3/projects/9f8845452eec4010b351362acd1186e8/users/4921f7950f2441d0b2b4154ee269cb17/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:24.585124 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae83c2-bbeb-4e9a-bc70-6d662358cf49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f8845452eec4010b351362acd1186e8, user_id=4921f7950f2441d0b2b4154ee269cb17, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:24.587802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae83c2-bbeb-4e9a-bc70-6d662358cf49 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:24.602513 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1022/2062] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:53:24 2022] PUT /identity/v3/projects/9f8845452eec4010b351362acd1186e8/users/4921f7950f2441d0b2b4154ee269cb17/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:53:24.613912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e6164-8805-4e9d-b7a4-f2acc7e5532a None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:24.614193 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e6164-8805-4e9d-b7a4-f2acc7e5532a None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:24.614340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e6164-8805-4e9d-b7a4-f2acc7e5532a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:24.648258 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-0c6e6164-8805-4e9d-b7a4-f2acc7e5532a None None] MFA Rules not processed for user `4921f7950f2441d0b2b4154ee269cb17`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:53:24.649666 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0c6e6164-8805-4e9d-b7a4-f2acc7e5532a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:24.723689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1041/2063] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:53:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:53:24.728376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae83c2-bbeb-4e9a-bc70-6d662358cf49 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:24.752514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:53:24.801945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae83c2-bbeb-4e9a-bc70-6d662358cf49 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:24.807994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d160c-1983-4bcb-8fbb-5a462d645538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:24.808653 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d160c-1983-4bcb-8fbb-5a462d645538 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:24.808868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d160c-1983-4bcb-8fbb-5a462d645538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:24.809160 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d160c-1983-4bcb-8fbb-5a462d645538 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:24.809898 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d160c-1983-4bcb-8fbb-5a462d645538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:24.812045 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d160c-1983-4bcb-8fbb-5a462d645538 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:24.839590 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1023/2064] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:53:24 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:53:24.843513 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e6164-8805-4e9d-b7a4-f2acc7e5532a None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:24.854983 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e6164-8805-4e9d-b7a4-f2acc7e5532a None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:24.913770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec9257-8fc8-4594-9a9b-7293cc24a668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:24.915520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec9257-8fc8-4594-9a9b-7293cc24a668 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:24.915855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec9257-8fc8-4594-9a9b-7293cc24a668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:24.916132 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec9257-8fc8-4594-9a9b-7293cc24a668 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:24.916763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec9257-8fc8-4594-9a9b-7293cc24a668 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:24.918737 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec9257-8fc8-4594-9a9b-7293cc24a668 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:24.940103 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1042/2065] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:53:24 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:53:24.944018 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d160c-1983-4bcb-8fbb-5a462d645538 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:24.955351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d160c-1983-4bcb-8fbb-5a462d645538 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:24.960173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31a16a-6815-4a6b-acd8-76712aab7499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:24.960786 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31a16a-6815-4a6b-acd8-76712aab7499 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:24.961568 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31a16a-6815-4a6b-acd8-76712aab7499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:24.961839 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31a16a-6815-4a6b-acd8-76712aab7499 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:24.962475 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31a16a-6815-4a6b-acd8-76712aab7499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:24.964095 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31a16a-6815-4a6b-acd8-76712aab7499 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:24.974894 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1024/2066] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:53:24.979709 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec9257-8fc8-4594-9a9b-7293cc24a668 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:24.984920 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:53:24.991562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec9257-8fc8-4594-9a9b-7293cc24a668 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:24.998409 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb57764-297e-4c58-a3d6-b3a757e8af60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:24.999071 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb57764-297e-4c58-a3d6-b3a757e8af60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:24.999361 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb57764-297e-4c58-a3d6-b3a757e8af60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:24.999654 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb57764-297e-4c58-a3d6-b3a757e8af60 admin admin] PATH_INFO: `/v3/projects/7232e296e47c47e4800411586586b035/users/54a658ece92d455e8ad44cfaba299ef6/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:25.015599 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb57764-297e-4c58-a3d6-b3a757e8af60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7232e296e47c47e4800411586586b035, user_id=54a658ece92d455e8ad44cfaba299ef6, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:25.018489 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb57764-297e-4c58-a3d6-b3a757e8af60 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:25.032120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1043/2067] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:53:24 2022] PUT /identity/v3/projects/7232e296e47c47e4800411586586b035/users/54a658ece92d455e8ad44cfaba299ef6/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:53:25.036156 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31a16a-6815-4a6b-acd8-76712aab7499 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:25.046924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31a16a-6815-4a6b-acd8-76712aab7499 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:25.108676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d03915-4045-4e85-95a1-cdc728716b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:25.109272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d03915-4045-4e85-95a1-cdc728716b5a admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:25.109553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d03915-4045-4e85-95a1-cdc728716b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:25.109827 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d03915-4045-4e85-95a1-cdc728716b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:25.110466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d03915-4045-4e85-95a1-cdc728716b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:25.112097 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d03915-4045-4e85-95a1-cdc728716b5a admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:25.122206 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1025/2068] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:53:25.126641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb57764-297e-4c58-a3d6-b3a757e8af60 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:25.138829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb57764-297e-4c58-a3d6-b3a757e8af60 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:25.144162 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371811e-ad8f-4cd9-89cc-b3c163a1cc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:25.144851 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371811e-ad8f-4cd9-89cc-b3c163a1cc02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:25.145155 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371811e-ad8f-4cd9-89cc-b3c163a1cc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:25.145429 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371811e-ad8f-4cd9-89cc-b3c163a1cc02 admin admin] PATH_INFO: `/v3/projects/7232e296e47c47e4800411586586b035/users/54a658ece92d455e8ad44cfaba299ef6/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:25.147354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371811e-ad8f-4cd9-89cc-b3c163a1cc02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7232e296e47c47e4800411586586b035, user_id=54a658ece92d455e8ad44cfaba299ef6, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:25.150597 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371811e-ad8f-4cd9-89cc-b3c163a1cc02 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:25.162285 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1044/2069] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:53:25 2022] PUT /identity/v3/projects/7232e296e47c47e4800411586586b035/users/54a658ece92d455e8ad44cfaba299ef6/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:53:25.173307 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67763f4-5baa-481c-9260-855239d2d5e1 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:25.173604 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67763f4-5baa-481c-9260-855239d2d5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:25.173948 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67763f4-5baa-481c-9260-855239d2d5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:25.204359 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-a67763f4-5baa-481c-9260-855239d2d5e1 None None] MFA Rules not processed for user `54a658ece92d455e8ad44cfaba299ef6`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:53:25.205572 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a67763f4-5baa-481c-9260-855239d2d5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:25.275496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1026/2070] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:53:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:53:25.286722 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371811e-ad8f-4cd9-89cc-b3c163a1cc02 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:25.288664 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-c371811e-ad8f-4cd9-89cc-b3c163a1cc02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:25.352817 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371811e-ad8f-4cd9-89cc-b3c163a1cc02 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:25.356391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7501e-6880-4c70-a946-111ea3cb848f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:25.356965 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7501e-6880-4c70-a946-111ea3cb848f service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:25.357236 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7501e-6880-4c70-a946-111ea3cb848f service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:25.357506 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7501e-6880-4c70-a946-111ea3cb848f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:25.366091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7501e-6880-4c70-a946-111ea3cb848f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:25.367539 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7501e-6880-4c70-a946-111ea3cb848f service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:25.377157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1045/2071] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:53:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:53:25.450151 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67763f4-5baa-481c-9260-855239d2d5e1 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:25.459622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67763f4-5baa-481c-9260-855239d2d5e1 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:25.463827 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691dd7b5-a3ef-40d3-a91d-07f8dcaf38a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:25.464426 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691dd7b5-a3ef-40d3-a91d-07f8dcaf38a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:25.464706 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691dd7b5-a3ef-40d3-a91d-07f8dcaf38a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:25.464977 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691dd7b5-a3ef-40d3-a91d-07f8dcaf38a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:25.482926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:53:25.525976 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691dd7b5-a3ef-40d3-a91d-07f8dcaf38a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:25.527471 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691dd7b5-a3ef-40d3-a91d-07f8dcaf38a1 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:25.537887 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1027/2072] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:53:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:53:26.710597 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7501e-6880-4c70-a946-111ea3cb848f service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:26.712742 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-48a7501e-6880-4c70-a946-111ea3cb848f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:26.785362 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7501e-6880-4c70-a946-111ea3cb848f service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:26.796405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62459215-a36c-4100-b81e-5240415861dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:26.797036 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62459215-a36c-4100-b81e-5240415861dd service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:26.797314 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62459215-a36c-4100-b81e-5240415861dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:26.797587 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62459215-a36c-4100-b81e-5240415861dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:26.799471 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-62459215-a36c-4100-b81e-5240415861dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:26.810514 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62459215-a36c-4100-b81e-5240415861dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:26.812073 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62459215-a36c-4100-b81e-5240415861dd service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:26.822647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1046/2073] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:53:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:53:33.707261 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691dd7b5-a3ef-40d3-a91d-07f8dcaf38a1 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:53:33.708879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-691dd7b5-a3ef-40d3-a91d-07f8dcaf38a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:33.814510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691dd7b5-a3ef-40d3-a91d-07f8dcaf38a1 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:53:33.820764 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1d6ab-0b90-4655-a01f-6d8b88b68cb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:53:33.821399 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1d6ab-0b90-4655-a01f-6d8b88b68cb1 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:53:33.821681 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1d6ab-0b90-4655-a01f-6d8b88b68cb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:53:33.821960 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1d6ab-0b90-4655-a01f-6d8b88b68cb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:53:33.823809 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-aac1d6ab-0b90-4655-a01f-6d8b88b68cb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:53:33.837438 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1d6ab-0b90-4655-a01f-6d8b88b68cb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:53:33.839186 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1d6ab-0b90-4655-a01f-6d8b88b68cb1 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:53:33.850933 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1028/2074] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:53:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:53:36.817032 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=70,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:53:43.845990 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=56 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:54:16.755010 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62459215-a36c-4100-b81e-5240415861dd service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:54:16.764562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62459215-a36c-4100-b81e-5240415861dd service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:54:16.768063 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e0c15-ec19-44da-8cc3-ed0034202a04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:54:16.768650 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e0c15-ec19-44da-8cc3-ed0034202a04 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:54:16.768922 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e0c15-ec19-44da-8cc3-ed0034202a04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:54:16.769185 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e0c15-ec19-44da-8cc3-ed0034202a04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:54:16.770919 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-920e0c15-ec19-44da-8cc3-ed0034202a04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:54:16.844522 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e0c15-ec19-44da-8cc3-ed0034202a04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:54:16.847074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e0c15-ec19-44da-8cc3-ed0034202a04 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:54:16.873227 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1047/2075] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:54:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:54:26.870114 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=36 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:26.517755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1d6ab-0b90-4655-a01f-6d8b88b68cb1 service nova] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:26.519251 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-aac1d6ab-0b90-4655-a01f-6d8b88b68cb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:26.530295 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1d6ab-0b90-4655-a01f-6d8b88b68cb1 service nova] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:26.534184 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d7e67-4abc-4ddc-8fea-e8a917f99b8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:26.534781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d7e67-4abc-4ddc-8fea-e8a917f99b8d service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:26.535053 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d7e67-4abc-4ddc-8fea-e8a917f99b8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:26.535320 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d7e67-4abc-4ddc-8fea-e8a917f99b8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:26.536991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-4f8d7e67-4abc-4ddc-8fea-e8a917f99b8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:26.597914 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d7e67-4abc-4ddc-8fea-e8a917f99b8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:26.599329 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d7e67-4abc-4ddc-8fea-e8a917f99b8d service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:26.608685 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1029/2076] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:55:33.284288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e0c15-ec19-44da-8cc3-ed0034202a04 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:33.294765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e0c15-ec19-44da-8cc3-ed0034202a04 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:33.298415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba9cfd-fa75-4402-9ed1-66f8c8334b41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:33.299059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba9cfd-fa75-4402-9ed1-66f8c8334b41 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:33.299338 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba9cfd-fa75-4402-9ed1-66f8c8334b41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:33.299642 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba9cfd-fa75-4402-9ed1-66f8c8334b41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:33.301465 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-beba9cfd-fa75-4402-9ed1-66f8c8334b41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:33.363016 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba9cfd-fa75-4402-9ed1-66f8c8334b41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:33.364475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba9cfd-fa75-4402-9ed1-66f8c8334b41 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:33.373949 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1048/2077] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:55:36.478247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d7e67-4abc-4ddc-8fea-e8a917f99b8d service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:36.479946 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-4f8d7e67-4abc-4ddc-8fea-e8a917f99b8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:36.533708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba9cfd-fa75-4402-9ed1-66f8c8334b41 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:36.536508 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-beba9cfd-fa75-4402-9ed1-66f8c8334b41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:36.553698 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba9cfd-fa75-4402-9ed1-66f8c8334b41 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:36.555048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d7e67-4abc-4ddc-8fea-e8a917f99b8d service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:36.558492 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebd313-84cf-49b0-ac75-8b61b3c951f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:36.559339 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebd313-84cf-49b0-ac75-8b61b3c951f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:36.559647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebd313-84cf-49b0-ac75-8b61b3c951f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:36.559919 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebd313-84cf-49b0-ac75-8b61b3c951f4 admin admin] PATH_INFO: `/v3/users/64e9cccb41a04aef83ab417783d82fcb` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:36.560161 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25130690-0d43-434a-b813-3cdbef6e650f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:36.560804 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25130690-0d43-434a-b813-3cdbef6e650f service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:36.561069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebd313-84cf-49b0-ac75-8b61b3c951f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64e9cccb41a04aef83ab417783d82fcb)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:36.561203 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25130690-0d43-434a-b813-3cdbef6e650f service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:36.561399 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25130690-0d43-434a-b813-3cdbef6e650f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:36.562981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebd313-84cf-49b0-ac75-8b61b3c951f4 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:36.563100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-25130690-0d43-434a-b813-3cdbef6e650f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:36.575376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25130690-0d43-434a-b813-3cdbef6e650f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:36.576983 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25130690-0d43-434a-b813-3cdbef6e650f service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:36.587667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1030/2078] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:55:36.618373 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-a6ebd313-84cf-49b0-ac75-8b61b3c951f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64e9cccb41a04aef83ab417783d82fcb'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:36.631441 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-a6ebd313-84cf-49b0-ac75-8b61b3c951f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64e9cccb41a04aef83ab417783d82fcb'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:36.639376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-a6ebd313-84cf-49b0-ac75-8b61b3c951f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64e9cccb41a04aef83ab417783d82fcb'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:36.640140 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-a6ebd313-84cf-49b0-ac75-8b61b3c951f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64e9cccb41a04aef83ab417783d82fcb'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:36.647475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-a6ebd313-84cf-49b0-ac75-8b61b3c951f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64e9cccb41a04aef83ab417783d82fcb'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:36.653962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1049/2079] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:55:36 2022] DELETE /identity/v3/users/64e9cccb41a04aef83ab417783d82fcb => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:55:36.829512 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25130690-0d43-434a-b813-3cdbef6e650f service placement] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:36.840602 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25130690-0d43-434a-b813-3cdbef6e650f service placement] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:36.900248 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128f06e-c409-4ae7-9fd5-2e396fa51e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:36.901037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128f06e-c409-4ae7-9fd5-2e396fa51e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:36.902876 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128f06e-c409-4ae7-9fd5-2e396fa51e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:36.903155 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128f06e-c409-4ae7-9fd5-2e396fa51e31 admin admin] PATH_INFO: `/v3/projects/62d044836c10479ca7c1975d4a64cbe8` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:36.904318 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3128f06e-c409-4ae7-9fd5-2e396fa51e31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62d044836c10479ca7c1975d4a64cbe8)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:36.906194 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3128f06e-c409-4ae7-9fd5-2e396fa51e31 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:36.939964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:36.957977 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3128f06e-c409-4ae7-9fd5-2e396fa51e31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62d044836c10479ca7c1975d4a64cbe8', 'request_id': 'req-3128f06e-c409-4ae7-9fd5-2e396fa51e31'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:36.965212 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3128f06e-c409-4ae7-9fd5-2e396fa51e31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62d044836c10479ca7c1975d4a64cbe8', 'request_id': 'req-3128f06e-c409-4ae7-9fd5-2e396fa51e31'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:36.971121 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3128f06e-c409-4ae7-9fd5-2e396fa51e31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62d044836c10479ca7c1975d4a64cbe8', 'request_id': 'req-3128f06e-c409-4ae7-9fd5-2e396fa51e31'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:36.982089 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3128f06e-c409-4ae7-9fd5-2e396fa51e31 admin admin] The token cache is being invalidate because project 62d044836c10479ca7c1975d4a64cbe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:55:36.982376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3128f06e-c409-4ae7-9fd5-2e396fa51e31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:36.983017 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3128f06e-c409-4ae7-9fd5-2e396fa51e31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:36.984532 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1031/2080] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:55:36 2022] DELETE /identity/v3/projects/62d044836c10479ca7c1975d4a64cbe8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:36.990577 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71545a-89c8-4af9-ae53-d387d30c0ba9 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:36.990864 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71545a-89c8-4af9-ae53-d387d30c0ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:36.991547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71545a-89c8-4af9-ae53-d387d30c0ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:37.010745 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-ca71545a-89c8-4af9-ae53-d387d30c0ba9 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:37.012046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ca71545a-89c8-4af9-ae53-d387d30c0ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:37.074096 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1050/2081] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:37.082059 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb7070-5fbb-4543-acf7-0807c940574c None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:37.082344 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb7070-5fbb-4543-acf7-0807c940574c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:37.082619 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb7070-5fbb-4543-acf7-0807c940574c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:37.102007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-6ddb7070-5fbb-4543-acf7-0807c940574c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:37.103150 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-6ddb7070-5fbb-4543-acf7-0807c940574c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:37.110942 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1032/2082] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:37.115648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71545a-89c8-4af9-ae53-d387d30c0ba9 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:37.125231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71545a-89c8-4af9-ae53-d387d30c0ba9 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:37.130411 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561adf39-ac2b-4091-be14-e84b4c98e38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:37.131051 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561adf39-ac2b-4091-be14-e84b4c98e38d admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:37.131326 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561adf39-ac2b-4091-be14-e84b4c98e38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:37.131620 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561adf39-ac2b-4091-be14-e84b4c98e38d admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:37.132255 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561adf39-ac2b-4091-be14-e84b4c98e38d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:55:37.132553 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561adf39-ac2b-4091-be14-e84b4c98e38d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:37.134261 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561adf39-ac2b-4091-be14-e84b4c98e38d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:37.136159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:37.151438 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1051/2083] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:55:37.152619 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb7070-5fbb-4543-acf7-0807c940574c None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:37.161837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb7070-5fbb-4543-acf7-0807c940574c None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:37.166408 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e1869-b5eb-4893-a1e2-cefceae943b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:37.166999 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e1869-b5eb-4893-a1e2-cefceae943b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:37.167278 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e1869-b5eb-4893-a1e2-cefceae943b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:37.167550 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e1869-b5eb-4893-a1e2-cefceae943b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:37.168210 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e1869-b5eb-4893-a1e2-cefceae943b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:37.170160 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e1869-b5eb-4893-a1e2-cefceae943b0 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:37.202497 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1033/2084] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:55:37 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:37.203742 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561adf39-ac2b-4091-be14-e84b4c98e38d admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:37.214453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561adf39-ac2b-4091-be14-e84b4c98e38d admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:37.272656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894de05-9a25-4a7d-86d0-aef02fa013fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:37.273214 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894de05-9a25-4a7d-86d0-aef02fa013fa admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:37.273493 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894de05-9a25-4a7d-86d0-aef02fa013fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:37.273806 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894de05-9a25-4a7d-86d0-aef02fa013fa admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:37.274462 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894de05-9a25-4a7d-86d0-aef02fa013fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:37.276468 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894de05-9a25-4a7d-86d0-aef02fa013fa admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:37.301411 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1052/2085] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:55:37 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:37.302598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e1869-b5eb-4893-a1e2-cefceae943b0 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:37.312874 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e1869-b5eb-4893-a1e2-cefceae943b0 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:37.318166 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37c900-31ed-46b7-83fe-7afc286a5250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:37.318774 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37c900-31ed-46b7-83fe-7afc286a5250 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:37.319054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37c900-31ed-46b7-83fe-7afc286a5250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:37.319360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37c900-31ed-46b7-83fe-7afc286a5250 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:37.320056 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37c900-31ed-46b7-83fe-7afc286a5250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:37.321647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37c900-31ed-46b7-83fe-7afc286a5250 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:37.332136 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1034/2086] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:37.336179 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894de05-9a25-4a7d-86d0-aef02fa013fa admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:37.346494 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894de05-9a25-4a7d-86d0-aef02fa013fa admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:37.351207 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f44986-578b-45b2-b7c2-ca9e685e8e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:37.351836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f44986-578b-45b2-b7c2-ca9e685e8e83 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:37.352168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f44986-578b-45b2-b7c2-ca9e685e8e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:37.352387 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f44986-578b-45b2-b7c2-ca9e685e8e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:37.353012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f44986-578b-45b2-b7c2-ca9e685e8e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:37.354612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f44986-578b-45b2-b7c2-ca9e685e8e83 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:37.365614 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1053/2087] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:37.369641 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37c900-31ed-46b7-83fe-7afc286a5250 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:37.379287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37c900-31ed-46b7-83fe-7afc286a5250 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:37.383812 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def61b43-d853-48a3-b9d8-8f7add3c8a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:37.384477 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def61b43-d853-48a3-b9d8-8f7add3c8a8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:37.384751 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def61b43-d853-48a3-b9d8-8f7add3c8a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:37.385020 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def61b43-d853-48a3-b9d8-8f7add3c8a8c admin admin] PATH_INFO: `/v3/projects/4318630ab7a1423b98ae82b2dc841f0b/users/a81d5ad2631242948bcc2791f15a065d/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:37.399305 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def61b43-d853-48a3-b9d8-8f7add3c8a8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4318630ab7a1423b98ae82b2dc841f0b, user_id=a81d5ad2631242948bcc2791f15a065d, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:37.401950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def61b43-d853-48a3-b9d8-8f7add3c8a8c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:37.414837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1035/2088] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:37 2022] PUT /identity/v3/projects/4318630ab7a1423b98ae82b2dc841f0b/users/a81d5ad2631242948bcc2791f15a065d/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:37.425682 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999157e-f5e5-4268-8626-b928929a03dd None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:37.425963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999157e-f5e5-4268-8626-b928929a03dd None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:37.426231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999157e-f5e5-4268-8626-b928929a03dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:37.460123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-d999157e-f5e5-4268-8626-b928929a03dd None None] MFA Rules not processed for user `a81d5ad2631242948bcc2791f15a065d`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:37.461459 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d999157e-f5e5-4268-8626-b928929a03dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:37.526356 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1054/2089] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:37.530889 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def61b43-d853-48a3-b9d8-8f7add3c8a8c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:37.594006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def61b43-d853-48a3-b9d8-8f7add3c8a8c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:37.599464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd86eb-d79a-4f73-9d65-dc7a35dc2a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:37.600089 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd86eb-d79a-4f73-9d65-dc7a35dc2a60 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:37.600367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd86eb-d79a-4f73-9d65-dc7a35dc2a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:37.600634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd86eb-d79a-4f73-9d65-dc7a35dc2a60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:37.601259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd86eb-d79a-4f73-9d65-dc7a35dc2a60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:37.603393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd86eb-d79a-4f73-9d65-dc7a35dc2a60 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:37.629284 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1036/2090] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:55:37 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:37.633408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999157e-f5e5-4268-8626-b928929a03dd None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:37.644171 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999157e-f5e5-4268-8626-b928929a03dd None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:37.701154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c6b25-f499-475b-a899-d1b1b0ff0ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:37.701818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c6b25-f499-475b-a899-d1b1b0ff0ede admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:37.702100 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c6b25-f499-475b-a899-d1b1b0ff0ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:37.702387 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c6b25-f499-475b-a899-d1b1b0ff0ede admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:37.703072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c6b25-f499-475b-a899-d1b1b0ff0ede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:37.705058 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c6b25-f499-475b-a899-d1b1b0ff0ede admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:37.715187 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:37.728622 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1055/2091] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:55:37 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:37.732811 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd86eb-d79a-4f73-9d65-dc7a35dc2a60 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:37.743273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd86eb-d79a-4f73-9d65-dc7a35dc2a60 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:37.748475 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853bc34-d8ce-4d39-a964-fed6f75bdb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:37.749081 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853bc34-d8ce-4d39-a964-fed6f75bdb19 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:37.749427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853bc34-d8ce-4d39-a964-fed6f75bdb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:37.749681 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853bc34-d8ce-4d39-a964-fed6f75bdb19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:37.750279 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4853bc34-d8ce-4d39-a964-fed6f75bdb19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:37.751938 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4853bc34-d8ce-4d39-a964-fed6f75bdb19 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:37.764501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1037/2092] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:37.769655 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c6b25-f499-475b-a899-d1b1b0ff0ede admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:37.779444 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c6b25-f499-475b-a899-d1b1b0ff0ede admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:37.784002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea6443-d682-4b99-a083-41110b3cf76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:37.785202 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea6443-d682-4b99-a083-41110b3cf76f admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:37.785472 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea6443-d682-4b99-a083-41110b3cf76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:37.785735 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea6443-d682-4b99-a083-41110b3cf76f admin admin] PATH_INFO: `/v3/projects/435796fdeb2d4d179dea9a94e2036fc0/users/8600ca7e2f2340d588dac05d7c8529df/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:37.799914 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea6443-d682-4b99-a083-41110b3cf76f admin admin] RBAC: Authorizing `identity:create_grant(project_id=435796fdeb2d4d179dea9a94e2036fc0, user_id=8600ca7e2f2340d588dac05d7c8529df, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:37.802546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea6443-d682-4b99-a083-41110b3cf76f admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:37.815058 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1056/2093] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:37 2022] PUT /identity/v3/projects/435796fdeb2d4d179dea9a94e2036fc0/users/8600ca7e2f2340d588dac05d7c8529df/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:37.819298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853bc34-d8ce-4d39-a964-fed6f75bdb19 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:37.829082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853bc34-d8ce-4d39-a964-fed6f75bdb19 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:37.849126 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:37.886388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90ac2e-0e33-4fbc-bdf1-f279e9920356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:37.886915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90ac2e-0e33-4fbc-bdf1-f279e9920356 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:37.887188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90ac2e-0e33-4fbc-bdf1-f279e9920356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:37.887456 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90ac2e-0e33-4fbc-bdf1-f279e9920356 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:37.888172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90ac2e-0e33-4fbc-bdf1-f279e9920356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:37.889773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90ac2e-0e33-4fbc-bdf1-f279e9920356 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:37.899716 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1038/2094] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:37.903972 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea6443-d682-4b99-a083-41110b3cf76f admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:37.915367 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea6443-d682-4b99-a083-41110b3cf76f admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:37.919941 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b88c0c-bc54-40bb-a766-759e106e1527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:37.920585 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b88c0c-bc54-40bb-a766-759e106e1527 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:37.920863 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b88c0c-bc54-40bb-a766-759e106e1527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:37.921132 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b88c0c-bc54-40bb-a766-759e106e1527 admin admin] PATH_INFO: `/v3/projects/435796fdeb2d4d179dea9a94e2036fc0/users/8600ca7e2f2340d588dac05d7c8529df/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:37.923033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b88c0c-bc54-40bb-a766-759e106e1527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=435796fdeb2d4d179dea9a94e2036fc0, user_id=8600ca7e2f2340d588dac05d7c8529df, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:37.925710 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b88c0c-bc54-40bb-a766-759e106e1527 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:37.935593 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1057/2095] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:37 2022] PUT /identity/v3/projects/435796fdeb2d4d179dea9a94e2036fc0/users/8600ca7e2f2340d588dac05d7c8529df/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:38.002563 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed758e1d-ceea-42ce-bf00-94748fa98e98 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:38.002888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed758e1d-ceea-42ce-bf00-94748fa98e98 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:38.003174 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed758e1d-ceea-42ce-bf00-94748fa98e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:38.032858 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-ed758e1d-ceea-42ce-bf00-94748fa98e98 None None] MFA Rules not processed for user `8600ca7e2f2340d588dac05d7c8529df`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:38.034074 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ed758e1d-ceea-42ce-bf00-94748fa98e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:38.106193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1039/2096] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:38.114911 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b88c0c-bc54-40bb-a766-759e106e1527 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:38.116603 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b4b88c0c-bc54-40bb-a766-759e106e1527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:38.172751 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b88c0c-bc54-40bb-a766-759e106e1527 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:38.176274 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba74cf1-89e7-4d83-a5bd-43dbb4d81515 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:38.176864 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba74cf1-89e7-4d83-a5bd-43dbb4d81515 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:38.177140 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba74cf1-89e7-4d83-a5bd-43dbb4d81515 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:38.177405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba74cf1-89e7-4d83-a5bd-43dbb4d81515 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:38.186148 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba74cf1-89e7-4d83-a5bd-43dbb4d81515 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:38.187584 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba74cf1-89e7-4d83-a5bd-43dbb4d81515 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:38.197223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1058/2097] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:55:38.331901 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed758e1d-ceea-42ce-bf00-94748fa98e98 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:38.333300 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ed758e1d-ceea-42ce-bf00-94748fa98e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:38.346120 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed758e1d-ceea-42ce-bf00-94748fa98e98 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:38.351920 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaed7b9-6830-4604-8bba-8bd0d35e5b39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:38.352526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaed7b9-6830-4604-8bba-8bd0d35e5b39 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:38.352801 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaed7b9-6830-4604-8bba-8bd0d35e5b39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:38.353067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaed7b9-6830-4604-8bba-8bd0d35e5b39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:38.412420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaed7b9-6830-4604-8bba-8bd0d35e5b39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:38.413869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaed7b9-6830-4604-8bba-8bd0d35e5b39 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:38.423362 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1040/2098] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:55:41.016216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba74cf1-89e7-4d83-a5bd-43dbb4d81515 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:41.078157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba74cf1-89e7-4d83-a5bd-43dbb4d81515 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:41.082686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fd276-dd24-4084-9b88-1287ebcb7c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:41.083311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fd276-dd24-4084-9b88-1287ebcb7c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:41.083614 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fd276-dd24-4084-9b88-1287ebcb7c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:41.083888 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fd276-dd24-4084-9b88-1287ebcb7c07 admin admin] PATH_INFO: `/v3/users/a81d5ad2631242948bcc2791f15a065d` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:41.085041 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fd276-dd24-4084-9b88-1287ebcb7c07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a81d5ad2631242948bcc2791f15a065d)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:41.087031 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fd276-dd24-4084-9b88-1287ebcb7c07 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:41.134726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-a65fd276-dd24-4084-9b88-1287ebcb7c07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a81d5ad2631242948bcc2791f15a065d'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:41.142972 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:41.148575 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-a65fd276-dd24-4084-9b88-1287ebcb7c07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a81d5ad2631242948bcc2791f15a065d'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:41.156790 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-a65fd276-dd24-4084-9b88-1287ebcb7c07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a81d5ad2631242948bcc2791f15a065d'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:41.157660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-a65fd276-dd24-4084-9b88-1287ebcb7c07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a81d5ad2631242948bcc2791f15a065d'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:41.165252 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-a65fd276-dd24-4084-9b88-1287ebcb7c07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a81d5ad2631242948bcc2791f15a065d'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:41.171065 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1059/2099] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:55:41 2022] DELETE /identity/v3/users/a81d5ad2631242948bcc2791f15a065d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:41.175534 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37914517-e232-41fc-8e7f-bbfc85fd6eda None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:41.176027 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37914517-e232-41fc-8e7f-bbfc85fd6eda None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:41.176158 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37914517-e232-41fc-8e7f-bbfc85fd6eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:41.196539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-37914517-e232-41fc-8e7f-bbfc85fd6eda None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:41.197752 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-37914517-e232-41fc-8e7f-bbfc85fd6eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:41.237693 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:41.258267 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1041/2100] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:41.268258 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fd276-dd24-4084-9b88-1287ebcb7c07 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:41.270200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-a65fd276-dd24-4084-9b88-1287ebcb7c07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:41.336326 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fd276-dd24-4084-9b88-1287ebcb7c07 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:41.339845 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a882d1d-a60b-4413-8613-d593e1a3ee34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:41.340448 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a882d1d-a60b-4413-8613-d593e1a3ee34 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:41.340727 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a882d1d-a60b-4413-8613-d593e1a3ee34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:41.340999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a882d1d-a60b-4413-8613-d593e1a3ee34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:41.350015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a882d1d-a60b-4413-8613-d593e1a3ee34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:41.351433 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a882d1d-a60b-4413-8613-d593e1a3ee34 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:41.360542 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1060/2101] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:55:41.663614 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37914517-e232-41fc-8e7f-bbfc85fd6eda None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:41.666511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-37914517-e232-41fc-8e7f-bbfc85fd6eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:41.688194 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37914517-e232-41fc-8e7f-bbfc85fd6eda None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:41.695440 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ac186-ae8a-4002-b182-61939b007ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:41.696485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ac186-ae8a-4002-b182-61939b007ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:41.696646 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ac186-ae8a-4002-b182-61939b007ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:41.697051 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ac186-ae8a-4002-b182-61939b007ad2 admin admin] PATH_INFO: `/v3/projects/4318630ab7a1423b98ae82b2dc841f0b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:41.699052 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ac186-ae8a-4002-b182-61939b007ad2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4318630ab7a1423b98ae82b2dc841f0b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:41.701151 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ac186-ae8a-4002-b182-61939b007ad2 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:41.773790 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-667ac186-ae8a-4002-b182-61939b007ad2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4318630ab7a1423b98ae82b2dc841f0b', 'request_id': 'req-667ac186-ae8a-4002-b182-61939b007ad2'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:41.784790 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-667ac186-ae8a-4002-b182-61939b007ad2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4318630ab7a1423b98ae82b2dc841f0b', 'request_id': 'req-667ac186-ae8a-4002-b182-61939b007ad2'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:41.793340 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-667ac186-ae8a-4002-b182-61939b007ad2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4318630ab7a1423b98ae82b2dc841f0b', 'request_id': 'req-667ac186-ae8a-4002-b182-61939b007ad2'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:41.809506 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-667ac186-ae8a-4002-b182-61939b007ad2 admin admin] The token cache is being invalidate because project 4318630ab7a1423b98ae82b2dc841f0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:55:41.809796 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-667ac186-ae8a-4002-b182-61939b007ad2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:41.810450 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-667ac186-ae8a-4002-b182-61939b007ad2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:41.811998 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1042/2102] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:55:41 2022] DELETE /identity/v3/projects/4318630ab7a1423b98ae82b2dc841f0b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:41.816246 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a882d1d-a60b-4413-8613-d593e1a3ee34 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:41.827152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a882d1d-a60b-4413-8613-d593e1a3ee34 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:41.886667 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97922dcb-296c-4c71-99eb-2b61f2827fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:41.887292 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97922dcb-296c-4c71-99eb-2b61f2827fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:41.887698 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97922dcb-296c-4c71-99eb-2b61f2827fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:41.887864 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97922dcb-296c-4c71-99eb-2b61f2827fa8 admin admin] PATH_INFO: `/v3/users/8600ca7e2f2340d588dac05d7c8529df` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:41.888963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97922dcb-296c-4c71-99eb-2b61f2827fa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8600ca7e2f2340d588dac05d7c8529df)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:41.891041 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97922dcb-296c-4c71-99eb-2b61f2827fa8 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:41.939122 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-97922dcb-296c-4c71-99eb-2b61f2827fa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8600ca7e2f2340d588dac05d7c8529df'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:41.951267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-97922dcb-296c-4c71-99eb-2b61f2827fa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8600ca7e2f2340d588dac05d7c8529df'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:41.955317 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:41.964402 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-97922dcb-296c-4c71-99eb-2b61f2827fa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8600ca7e2f2340d588dac05d7c8529df'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:41.965020 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-97922dcb-296c-4c71-99eb-2b61f2827fa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8600ca7e2f2340d588dac05d7c8529df'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:41.972494 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-97922dcb-296c-4c71-99eb-2b61f2827fa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8600ca7e2f2340d588dac05d7c8529df'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:41.978286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1061/2103] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:55:41 2022] DELETE /identity/v3/users/8600ca7e2f2340d588dac05d7c8529df => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:42.251436 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ac186-ae8a-4002-b182-61939b007ad2 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:42.253251 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-667ac186-ae8a-4002-b182-61939b007ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:42.269197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ac186-ae8a-4002-b182-61939b007ad2 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:42.336896 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1b14a-afc5-48a6-85fe-2254b93d7aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:42.337534 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1b14a-afc5-48a6-85fe-2254b93d7aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:42.337813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1b14a-afc5-48a6-85fe-2254b93d7aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:42.338085 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1b14a-afc5-48a6-85fe-2254b93d7aec admin admin] PATH_INFO: `/v3/projects/435796fdeb2d4d179dea9a94e2036fc0` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:42.339211 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1b14a-afc5-48a6-85fe-2254b93d7aec admin admin] RBAC: Authorizing `identity:delete_project(project_id=435796fdeb2d4d179dea9a94e2036fc0)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:42.341149 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1b14a-afc5-48a6-85fe-2254b93d7aec admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:42.387029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-99a1b14a-afc5-48a6-85fe-2254b93d7aec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '435796fdeb2d4d179dea9a94e2036fc0', 'request_id': 'req-99a1b14a-afc5-48a6-85fe-2254b93d7aec'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:42.394117 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-99a1b14a-afc5-48a6-85fe-2254b93d7aec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '435796fdeb2d4d179dea9a94e2036fc0', 'request_id': 'req-99a1b14a-afc5-48a6-85fe-2254b93d7aec'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:42.399868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-99a1b14a-afc5-48a6-85fe-2254b93d7aec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '435796fdeb2d4d179dea9a94e2036fc0', 'request_id': 'req-99a1b14a-afc5-48a6-85fe-2254b93d7aec'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:42.406889 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:42.412042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-99a1b14a-afc5-48a6-85fe-2254b93d7aec admin admin] The token cache is being invalidate because project 435796fdeb2d4d179dea9a94e2036fc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:55:42.412517 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-99a1b14a-afc5-48a6-85fe-2254b93d7aec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:42.413525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-99a1b14a-afc5-48a6-85fe-2254b93d7aec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:42.415243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1043/2104] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:55:42 2022] DELETE /identity/v3/projects/435796fdeb2d4d179dea9a94e2036fc0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:42.420047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e29ae4-e001-4555-ae18-d84d0a788d8f None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:42.420533 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e29ae4-e001-4555-ae18-d84d0a788d8f None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:42.420640 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e29ae4-e001-4555-ae18-d84d0a788d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:42.441080 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-96e29ae4-e001-4555-ae18-d84d0a788d8f None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:42.442271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-96e29ae4-e001-4555-ae18-d84d0a788d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:42.498592 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1062/2105] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:55:42.506943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd4ef5-b54a-44f7-a39c-4c649fa1646c None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:42.507229 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd4ef5-b54a-44f7-a39c-4c649fa1646c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:42.507500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd4ef5-b54a-44f7-a39c-4c649fa1646c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:42.526796 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-88fd4ef5-b54a-44f7-a39c-4c649fa1646c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:42.528016 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-88fd4ef5-b54a-44f7-a39c-4c649fa1646c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:42.536184 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1044/2106] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:42.540144 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e29ae4-e001-4555-ae18-d84d0a788d8f None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:42.550308 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e29ae4-e001-4555-ae18-d84d0a788d8f None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:42.554811 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d438940-b81c-4a7c-8be9-dcf92bcb2eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:42.555399 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d438940-b81c-4a7c-8be9-dcf92bcb2eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:42.555713 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d438940-b81c-4a7c-8be9-dcf92bcb2eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:42.555986 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d438940-b81c-4a7c-8be9-dcf92bcb2eb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:42.556612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d438940-b81c-4a7c-8be9-dcf92bcb2eb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:55:42.556912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d438940-b81c-4a7c-8be9-dcf92bcb2eb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:42.558536 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d438940-b81c-4a7c-8be9-dcf92bcb2eb3 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:42.570509 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1063/2107] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:55:42.574723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd4ef5-b54a-44f7-a39c-4c649fa1646c None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:42.583860 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd4ef5-b54a-44f7-a39c-4c649fa1646c None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:42.588390 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1506f-055b-4692-8389-da616b565f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:42.588976 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1506f-055b-4692-8389-da616b565f82 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:42.589252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1506f-055b-4692-8389-da616b565f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:42.589520 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1506f-055b-4692-8389-da616b565f82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:42.590136 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1506f-055b-4692-8389-da616b565f82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:42.592127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1506f-055b-4692-8389-da616b565f82 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:42.621533 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1045/2108] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:55:42 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:42.625643 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d438940-b81c-4a7c-8be9-dcf92bcb2eb3 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:42.635988 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d438940-b81c-4a7c-8be9-dcf92bcb2eb3 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:42.692895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2145a1-8726-4119-bb7d-f28c62ef9a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:42.693503 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2145a1-8726-4119-bb7d-f28c62ef9a85 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:42.693774 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2145a1-8726-4119-bb7d-f28c62ef9a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:42.694130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2145a1-8726-4119-bb7d-f28c62ef9a85 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:42.694818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2145a1-8726-4119-bb7d-f28c62ef9a85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:42.696727 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2145a1-8726-4119-bb7d-f28c62ef9a85 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:42.722022 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1064/2109] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:55:42 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:42.724332 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1506f-055b-4692-8389-da616b565f82 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:42.734953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1506f-055b-4692-8389-da616b565f82 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:42.740342 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76320791-cc2b-44ae-9533-5aee21d7fdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:42.740951 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76320791-cc2b-44ae-9533-5aee21d7fdb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:42.741234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76320791-cc2b-44ae-9533-5aee21d7fdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:42.741506 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76320791-cc2b-44ae-9533-5aee21d7fdb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:42.742144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76320791-cc2b-44ae-9533-5aee21d7fdb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:42.743792 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76320791-cc2b-44ae-9533-5aee21d7fdb5 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:42.753888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1046/2110] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:42.757817 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2145a1-8726-4119-bb7d-f28c62ef9a85 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:42.767888 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2145a1-8726-4119-bb7d-f28c62ef9a85 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:42.772537 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e33432-9179-43f0-9df9-2bf0037372a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:42.773125 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e33432-9179-43f0-9df9-2bf0037372a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:42.773403 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e33432-9179-43f0-9df9-2bf0037372a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:42.773670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e33432-9179-43f0-9df9-2bf0037372a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:42.774294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e33432-9179-43f0-9df9-2bf0037372a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:42.775920 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e33432-9179-43f0-9df9-2bf0037372a3 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:42.786720 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1065/2111] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:42.790722 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76320791-cc2b-44ae-9533-5aee21d7fdb5 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:42.801093 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76320791-cc2b-44ae-9533-5aee21d7fdb5 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:42.805644 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c021a-741a-499e-bb0b-39874b7b6347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:42.806284 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c021a-741a-499e-bb0b-39874b7b6347 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:42.806559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c021a-741a-499e-bb0b-39874b7b6347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:42.806832 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c021a-741a-499e-bb0b-39874b7b6347 admin admin] PATH_INFO: `/v3/projects/0772248ea0574d01a41bb4e651ea3738/users/f738d806fb8c47cd95cc017d2ccfe08d/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:42.823858 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c021a-741a-499e-bb0b-39874b7b6347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0772248ea0574d01a41bb4e651ea3738, user_id=f738d806fb8c47cd95cc017d2ccfe08d, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:42.826497 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c021a-741a-499e-bb0b-39874b7b6347 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:42.840244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1047/2112] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:42 2022] PUT /identity/v3/projects/0772248ea0574d01a41bb4e651ea3738/users/f738d806fb8c47cd95cc017d2ccfe08d/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:55:42.851125 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db249e-69a6-4083-8c07-e9451c919262 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:42.851408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db249e-69a6-4083-8c07-e9451c919262 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:42.851725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db249e-69a6-4083-8c07-e9451c919262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:42.881489 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-03db249e-69a6-4083-8c07-e9451c919262 None None] MFA Rules not processed for user `f738d806fb8c47cd95cc017d2ccfe08d`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:42.882690 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-03db249e-69a6-4083-8c07-e9451c919262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:42.927287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:42.953649 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1066/2113] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:42.958874 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c021a-741a-499e-bb0b-39874b7b6347 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:43.035209 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c021a-741a-499e-bb0b-39874b7b6347 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:43.039941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082aa8a-18bc-4594-9328-26d2bf2c62c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:43.040608 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082aa8a-18bc-4594-9328-26d2bf2c62c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:43.041049 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082aa8a-18bc-4594-9328-26d2bf2c62c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:43.041379 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082aa8a-18bc-4594-9328-26d2bf2c62c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:43.042065 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082aa8a-18bc-4594-9328-26d2bf2c62c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:43.044029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082aa8a-18bc-4594-9328-26d2bf2c62c3 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:43.085359 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1048/2114] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:55:42 2022] POST /identity/v3/projects => generated 379 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:43.089339 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db249e-69a6-4083-8c07-e9451c919262 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:43.102756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db249e-69a6-4083-8c07-e9451c919262 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:43.167080 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7e9da-1870-4286-9382-769e81c57858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:43.167854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7e9da-1870-4286-9382-769e81c57858 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:43.168136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7e9da-1870-4286-9382-769e81c57858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:43.168536 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7e9da-1870-4286-9382-769e81c57858 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:43.169030 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7e9da-1870-4286-9382-769e81c57858 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:43.170942 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7e9da-1870-4286-9382-769e81c57858 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:43.194328 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1067/2115] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:55:43 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:43.198551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082aa8a-18bc-4594-9328-26d2bf2c62c3 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:43.213200 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082aa8a-18bc-4594-9328-26d2bf2c62c3 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:43.220237 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75265488-2c6c-4338-bf91-4649c35fa24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:43.220881 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265488-2c6c-4338-bf91-4649c35fa24c admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:43.221158 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265488-2c6c-4338-bf91-4649c35fa24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:43.221533 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265488-2c6c-4338-bf91-4649c35fa24c admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:43.222910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75265488-2c6c-4338-bf91-4649c35fa24c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:43.224709 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75265488-2c6c-4338-bf91-4649c35fa24c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:43.237608 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1049/2116] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:43.241719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7e9da-1870-4286-9382-769e81c57858 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:43.252695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7e9da-1870-4286-9382-769e81c57858 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:43.257445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12b484-c444-42e0-84e3-3d4ba03b0a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:43.258110 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12b484-c444-42e0-84e3-3d4ba03b0a16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:43.258387 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12b484-c444-42e0-84e3-3d4ba03b0a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:43.258661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12b484-c444-42e0-84e3-3d4ba03b0a16 admin admin] PATH_INFO: `/v3/projects/c76d33776b084813b0ce5bf921d25b06/users/8bf16da90bdc4e29a450d227b01fdc59/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:43.274801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12b484-c444-42e0-84e3-3d4ba03b0a16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c76d33776b084813b0ce5bf921d25b06, user_id=8bf16da90bdc4e29a450d227b01fdc59, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:43.280709 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12b484-c444-42e0-84e3-3d4ba03b0a16 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:43.293923 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1068/2117] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:43 2022] PUT /identity/v3/projects/c76d33776b084813b0ce5bf921d25b06/users/8bf16da90bdc4e29a450d227b01fdc59/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:43.297841 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75265488-2c6c-4338-bf91-4649c35fa24c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:43.307963 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75265488-2c6c-4338-bf91-4649c35fa24c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:43.364324 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df4e68-b017-4463-aa03-cced7b9044c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:43.364904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df4e68-b017-4463-aa03-cced7b9044c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:43.365277 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df4e68-b017-4463-aa03-cced7b9044c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:43.365450 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df4e68-b017-4463-aa03-cced7b9044c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:43.366106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df4e68-b017-4463-aa03-cced7b9044c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:43.367730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df4e68-b017-4463-aa03-cced7b9044c7 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:43.381019 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1050/2118] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:43.382265 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12b484-c444-42e0-84e3-3d4ba03b0a16 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:43.393985 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12b484-c444-42e0-84e3-3d4ba03b0a16 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:43.398524 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ebd67-7327-4d39-924d-e25f83481941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:43.399216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ebd67-7327-4d39-924d-e25f83481941 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:43.399479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ebd67-7327-4d39-924d-e25f83481941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:43.399785 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ebd67-7327-4d39-924d-e25f83481941 admin admin] PATH_INFO: `/v3/projects/c76d33776b084813b0ce5bf921d25b06/users/8bf16da90bdc4e29a450d227b01fdc59/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:43.401650 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ebd67-7327-4d39-924d-e25f83481941 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c76d33776b084813b0ce5bf921d25b06, user_id=8bf16da90bdc4e29a450d227b01fdc59, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:43.404331 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ebd67-7327-4d39-924d-e25f83481941 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:43.415038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1069/2119] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:43 2022] PUT /identity/v3/projects/c76d33776b084813b0ce5bf921d25b06/users/8bf16da90bdc4e29a450d227b01fdc59/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:43.426136 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43203d-1989-482c-bb6b-ef0d74a4bc42 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:43.426444 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43203d-1989-482c-bb6b-ef0d74a4bc42 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:43.426723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43203d-1989-482c-bb6b-ef0d74a4bc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:43.435224 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:43.457695 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-7d43203d-1989-482c-bb6b-ef0d74a4bc42 None None] MFA Rules not processed for user `8bf16da90bdc4e29a450d227b01fdc59`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:43.458973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-7d43203d-1989-482c-bb6b-ef0d74a4bc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:43.529464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1051/2120] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:43 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:55:43.542769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ebd67-7327-4d39-924d-e25f83481941 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:43.544510 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-931ebd67-7327-4d39-924d-e25f83481941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:43.602352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ebd67-7327-4d39-924d-e25f83481941 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:43.606385 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47e4fd-804c-4266-9ba2-1538ae57d05b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:43.606988 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47e4fd-804c-4266-9ba2-1538ae57d05b service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:43.607263 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47e4fd-804c-4266-9ba2-1538ae57d05b service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:43.607573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47e4fd-804c-4266-9ba2-1538ae57d05b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:43.617013 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47e4fd-804c-4266-9ba2-1538ae57d05b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:43.618432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47e4fd-804c-4266-9ba2-1538ae57d05b service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:43.628461 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1070/2121] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:43 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:55:43.964361 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43203d-1989-482c-bb6b-ef0d74a4bc42 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:43.974031 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43203d-1989-482c-bb6b-ef0d74a4bc42 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:43.978578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e9c90-482d-4e28-8d64-118322c0caf1 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bf16da90bdc4e29a450d227b01fdc59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c76d33776b084813b0ce5bf921d25b06', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:43.979173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e9c90-482d-4e28-8d64-118322c0caf1 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:43.979460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e9c90-482d-4e28-8d64-118322c0caf1 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:43.979766 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e9c90-482d-4e28-8d64-118322c0caf1 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:43.980410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e9c90-482d-4e28-8d64-118322c0caf1 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:55:43.980714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e9c90-482d-4e28-8d64-118322c0caf1 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:43.982302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e9c90-482d-4e28-8d64-118322c0caf1 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:43.994124 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1052/2122] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:55:43.998327 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47e4fd-804c-4266-9ba2-1538ae57d05b service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:44.008049 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47e4fd-804c-4266-9ba2-1538ae57d05b service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:44.013063 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b31108-5090-4cf8-bedb-1b9b47417cbc tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bf16da90bdc4e29a450d227b01fdc59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c76d33776b084813b0ce5bf921d25b06', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:44.013664 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b31108-5090-4cf8-bedb-1b9b47417cbc tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:44.013948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b31108-5090-4cf8-bedb-1b9b47417cbc tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:44.014225 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b31108-5090-4cf8-bedb-1b9b47417cbc tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:44.014885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b31108-5090-4cf8-bedb-1b9b47417cbc tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:44.016872 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b31108-5090-4cf8-bedb-1b9b47417cbc tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:44.043015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1071/2123] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:55:43 2022] POST /identity/v3/projects => generated 354 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:55:44.047085 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e9c90-482d-4e28-8d64-118322c0caf1 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:44.057731 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e9c90-482d-4e28-8d64-118322c0caf1 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:44.111637 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971551a-bde1-461f-a0b6-9699b25c6eda tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bf16da90bdc4e29a450d227b01fdc59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c76d33776b084813b0ce5bf921d25b06', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:44.112789 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971551a-bde1-461f-a0b6-9699b25c6eda tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:44.113072 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971551a-bde1-461f-a0b6-9699b25c6eda tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:44.113348 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971551a-bde1-461f-a0b6-9699b25c6eda tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:44.113978 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971551a-bde1-461f-a0b6-9699b25c6eda tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:55:44.114282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971551a-bde1-461f-a0b6-9699b25c6eda tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:44.115878 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971551a-bde1-461f-a0b6-9699b25c6eda tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:44.127560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1053/2124] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:55:44.166383 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b31108-5090-4cf8-bedb-1b9b47417cbc tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:44.178052 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b31108-5090-4cf8-bedb-1b9b47417cbc tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:44.183851 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30caaad-5a21-4a53-b95a-7c083a2cbb29 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bf16da90bdc4e29a450d227b01fdc59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c76d33776b084813b0ce5bf921d25b06', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:44.184479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30caaad-5a21-4a53-b95a-7c083a2cbb29 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:44.184756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30caaad-5a21-4a53-b95a-7c083a2cbb29 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:44.185023 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30caaad-5a21-4a53-b95a-7c083a2cbb29 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] PATH_INFO: `/v3/projects/8801d90c2b8e47b4930451ae66bc4507` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:44.186166 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30caaad-5a21-4a53-b95a-7c083a2cbb29 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8801d90c2b8e47b4930451ae66bc4507)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:44.188205 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30caaad-5a21-4a53-b95a-7c083a2cbb29 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:44.201067 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:44.241046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c30caaad-5a21-4a53-b95a-7c083a2cbb29 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8801d90c2b8e47b4930451ae66bc4507', 'request_id': 'req-c30caaad-5a21-4a53-b95a-7c083a2cbb29'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:44.249131 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c30caaad-5a21-4a53-b95a-7c083a2cbb29 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8801d90c2b8e47b4930451ae66bc4507', 'request_id': 'req-c30caaad-5a21-4a53-b95a-7c083a2cbb29'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:44.257069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c30caaad-5a21-4a53-b95a-7c083a2cbb29 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8801d90c2b8e47b4930451ae66bc4507', 'request_id': 'req-c30caaad-5a21-4a53-b95a-7c083a2cbb29'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:44.268690 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c30caaad-5a21-4a53-b95a-7c083a2cbb29 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] The token cache is being invalidate because project 8801d90c2b8e47b4930451ae66bc4507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:55:44.268982 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c30caaad-5a21-4a53-b95a-7c083a2cbb29 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:44.269746 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c30caaad-5a21-4a53-b95a-7c083a2cbb29 tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:44.271258 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1072/2125] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:55:44 2022] DELETE /identity/v3/projects/8801d90c2b8e47b4930451ae66bc4507 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:44.551451 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971551a-bde1-461f-a0b6-9699b25c6eda tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:44.554975 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-4971551a-bde1-461f-a0b6-9699b25c6eda tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:44.640227 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971551a-bde1-461f-a0b6-9699b25c6eda tempest-VolumeQuotaClassesTest-994806841 tempest-VolumeQuotaClassesTest-994806841-project-admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:44.652446 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:44.700481 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218cd6e5-dfda-46d9-a5a7-dedfa74ac801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:44.701117 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cd6e5-dfda-46d9-a5a7-dedfa74ac801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:44.701399 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cd6e5-dfda-46d9-a5a7-dedfa74ac801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:44.701673 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cd6e5-dfda-46d9-a5a7-dedfa74ac801 admin admin] PATH_INFO: `/v3/users/f738d806fb8c47cd95cc017d2ccfe08d` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:44.702801 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218cd6e5-dfda-46d9-a5a7-dedfa74ac801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f738d806fb8c47cd95cc017d2ccfe08d)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:44.704775 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218cd6e5-dfda-46d9-a5a7-dedfa74ac801 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:44.762041 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-218cd6e5-dfda-46d9-a5a7-dedfa74ac801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f738d806fb8c47cd95cc017d2ccfe08d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:44.762700 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-218cd6e5-dfda-46d9-a5a7-dedfa74ac801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f738d806fb8c47cd95cc017d2ccfe08d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:44.774528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-218cd6e5-dfda-46d9-a5a7-dedfa74ac801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f738d806fb8c47cd95cc017d2ccfe08d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:44.782469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-218cd6e5-dfda-46d9-a5a7-dedfa74ac801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f738d806fb8c47cd95cc017d2ccfe08d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:44.783090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-218cd6e5-dfda-46d9-a5a7-dedfa74ac801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f738d806fb8c47cd95cc017d2ccfe08d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:44.796542 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1054/2126] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:55:44 2022] DELETE /identity/v3/users/f738d806fb8c47cd95cc017d2ccfe08d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:44.801779 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ac9a2-0648-473a-ba7c-4789305be0f2 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:44.802279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ac9a2-0648-473a-ba7c-4789305be0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:44.802387 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ac9a2-0648-473a-ba7c-4789305be0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:44.820897 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-a86ac9a2-0648-473a-ba7c-4789305be0f2 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:44.822074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-a86ac9a2-0648-473a-ba7c-4789305be0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:44.886598 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1073/2127] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:55:44.896501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218cd6e5-dfda-46d9-a5a7-dedfa74ac801 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:44.898126 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-218cd6e5-dfda-46d9-a5a7-dedfa74ac801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:44.955847 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218cd6e5-dfda-46d9-a5a7-dedfa74ac801 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:44.959725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5f7c0-9a46-43b0-a650-3c835f6a07e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:44.960326 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5f7c0-9a46-43b0-a650-3c835f6a07e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:44.960605 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5f7c0-9a46-43b0-a650-3c835f6a07e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:44.960880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5f7c0-9a46-43b0-a650-3c835f6a07e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:44.970125 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5f7c0-9a46-43b0-a650-3c835f6a07e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:44.972077 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5f7c0-9a46-43b0-a650-3c835f6a07e8 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:44.981731 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1055/2128] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:55:45.267851 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ac9a2-0648-473a-ba7c-4789305be0f2 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:45.269885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-a86ac9a2-0648-473a-ba7c-4789305be0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:45.287331 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ac9a2-0648-473a-ba7c-4789305be0f2 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:45.292060 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98dfdb1-a6eb-4008-a412-b43bef8f512a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:45.292687 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98dfdb1-a6eb-4008-a412-b43bef8f512a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:45.292959 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98dfdb1-a6eb-4008-a412-b43bef8f512a admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:45.293251 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98dfdb1-a6eb-4008-a412-b43bef8f512a admin admin] PATH_INFO: `/v3/projects/0772248ea0574d01a41bb4e651ea3738` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:45.294310 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98dfdb1-a6eb-4008-a412-b43bef8f512a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0772248ea0574d01a41bb4e651ea3738)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:45.296315 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98dfdb1-a6eb-4008-a412-b43bef8f512a admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:45.343045 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d98dfdb1-a6eb-4008-a412-b43bef8f512a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0772248ea0574d01a41bb4e651ea3738', 'request_id': 'req-d98dfdb1-a6eb-4008-a412-b43bef8f512a'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:45.350179 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d98dfdb1-a6eb-4008-a412-b43bef8f512a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0772248ea0574d01a41bb4e651ea3738', 'request_id': 'req-d98dfdb1-a6eb-4008-a412-b43bef8f512a'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:45.357282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d98dfdb1-a6eb-4008-a412-b43bef8f512a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0772248ea0574d01a41bb4e651ea3738', 'request_id': 'req-d98dfdb1-a6eb-4008-a412-b43bef8f512a'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:45.368096 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d98dfdb1-a6eb-4008-a412-b43bef8f512a admin admin] The token cache is being invalidate because project 0772248ea0574d01a41bb4e651ea3738 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:55:45.368460 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d98dfdb1-a6eb-4008-a412-b43bef8f512a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:45.369109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d98dfdb1-a6eb-4008-a412-b43bef8f512a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:45.370691 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1074/2129] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:55:45 2022] DELETE /identity/v3/projects/0772248ea0574d01a41bb4e651ea3738 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:45.374901 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5f7c0-9a46-43b0-a650-3c835f6a07e8 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:45.384725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5f7c0-9a46-43b0-a650-3c835f6a07e8 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:45.425634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:45.445778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d502f00-a58e-436b-8888-005d8b3ca678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:45.446398 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d502f00-a58e-436b-8888-005d8b3ca678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:45.446677 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d502f00-a58e-436b-8888-005d8b3ca678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:45.446947 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d502f00-a58e-436b-8888-005d8b3ca678 admin admin] PATH_INFO: `/v3/users/8bf16da90bdc4e29a450d227b01fdc59` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:45.448090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d502f00-a58e-436b-8888-005d8b3ca678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf16da90bdc4e29a450d227b01fdc59)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:45.449986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d502f00-a58e-436b-8888-005d8b3ca678 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:45.497209 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3d502f00-a58e-436b-8888-005d8b3ca678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf16da90bdc4e29a450d227b01fdc59'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:45.497939 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3d502f00-a58e-436b-8888-005d8b3ca678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf16da90bdc4e29a450d227b01fdc59'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:45.511214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3d502f00-a58e-436b-8888-005d8b3ca678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf16da90bdc4e29a450d227b01fdc59'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:45.520699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3d502f00-a58e-436b-8888-005d8b3ca678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf16da90bdc4e29a450d227b01fdc59'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:45.521318 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-3d502f00-a58e-436b-8888-005d8b3ca678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf16da90bdc4e29a450d227b01fdc59'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:45.548601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1056/2130] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:55:45 2022] DELETE /identity/v3/users/8bf16da90bdc4e29a450d227b01fdc59 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:45.878598 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98dfdb1-a6eb-4008-a412-b43bef8f512a admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:45.880229 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d98dfdb1-a6eb-4008-a412-b43bef8f512a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:45.901665 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98dfdb1-a6eb-4008-a412-b43bef8f512a admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:45.927551 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:45.977940 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aecd59d-c163-480f-9ec8-80d8e31baca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:45.978810 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecd59d-c163-480f-9ec8-80d8e31baca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:45.979095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecd59d-c163-480f-9ec8-80d8e31baca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:45.979437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecd59d-c163-480f-9ec8-80d8e31baca8 admin admin] PATH_INFO: `/v3/projects/c76d33776b084813b0ce5bf921d25b06` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:45.981060 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecd59d-c163-480f-9ec8-80d8e31baca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c76d33776b084813b0ce5bf921d25b06)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:45.983204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecd59d-c163-480f-9ec8-80d8e31baca8 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:46.051388 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2aecd59d-c163-480f-9ec8-80d8e31baca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c76d33776b084813b0ce5bf921d25b06', 'request_id': 'req-2aecd59d-c163-480f-9ec8-80d8e31baca8'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:46.060526 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2aecd59d-c163-480f-9ec8-80d8e31baca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c76d33776b084813b0ce5bf921d25b06', 'request_id': 'req-2aecd59d-c163-480f-9ec8-80d8e31baca8'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:46.082710 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2aecd59d-c163-480f-9ec8-80d8e31baca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c76d33776b084813b0ce5bf921d25b06', 'request_id': 'req-2aecd59d-c163-480f-9ec8-80d8e31baca8'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:46.096588 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2aecd59d-c163-480f-9ec8-80d8e31baca8 admin admin] The token cache is being invalidate because project c76d33776b084813b0ce5bf921d25b06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:55:46.096879 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2aecd59d-c163-480f-9ec8-80d8e31baca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:46.100348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-2aecd59d-c163-480f-9ec8-80d8e31baca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:46.102954 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1075/2131] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:55:45 2022] DELETE /identity/v3/projects/c76d33776b084813b0ce5bf921d25b06 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:46.108760 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e916a4-60d6-4cee-8e51-14e67fdb96ec None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:46.109243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e916a4-60d6-4cee-8e51-14e67fdb96ec None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:46.109592 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e916a4-60d6-4cee-8e51-14e67fdb96ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:46.137578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-53e916a4-60d6-4cee-8e51-14e67fdb96ec None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:46.139251 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-53e916a4-60d6-4cee-8e51-14e67fdb96ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:46.216748 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1057/2132] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:46.225266 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eba14d-b7d5-426b-a209-bc2211508e76 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:46.225699 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eba14d-b7d5-426b-a209-bc2211508e76 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:46.225972 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eba14d-b7d5-426b-a209-bc2211508e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:46.248599 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-d0eba14d-b7d5-426b-a209-bc2211508e76 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:46.249984 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d0eba14d-b7d5-426b-a209-bc2211508e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:46.258542 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1076/2133] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:46.263425 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e916a4-60d6-4cee-8e51-14e67fdb96ec None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:46.273456 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e916a4-60d6-4cee-8e51-14e67fdb96ec None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:46.277982 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17b28f-a08a-457b-8d8c-5654a7258655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:46.278580 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17b28f-a08a-457b-8d8c-5654a7258655 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:46.278860 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17b28f-a08a-457b-8d8c-5654a7258655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:46.279133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17b28f-a08a-457b-8d8c-5654a7258655 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:46.279811 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d17b28f-a08a-457b-8d8c-5654a7258655 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:55:46.280119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d17b28f-a08a-457b-8d8c-5654a7258655 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:46.281715 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d17b28f-a08a-457b-8d8c-5654a7258655 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:46.293571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1058/2134] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:55:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:55:46.297718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eba14d-b7d5-426b-a209-bc2211508e76 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:46.307406 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eba14d-b7d5-426b-a209-bc2211508e76 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:46.311987 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797a578-4dd6-4084-a0e0-b4d288398959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:46.312579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797a578-4dd6-4084-a0e0-b4d288398959 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:46.312857 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797a578-4dd6-4084-a0e0-b4d288398959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:46.313176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797a578-4dd6-4084-a0e0-b4d288398959 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:46.313791 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797a578-4dd6-4084-a0e0-b4d288398959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:46.315751 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797a578-4dd6-4084-a0e0-b4d288398959 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:46.342164 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1077/2135] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:55:46 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:46.346770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17b28f-a08a-457b-8d8c-5654a7258655 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:46.356538 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17b28f-a08a-457b-8d8c-5654a7258655 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:46.399232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:46.412985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72e5f3-aeec-47d9-86ee-3ff15c9c700b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:46.413589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72e5f3-aeec-47d9-86ee-3ff15c9c700b admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:46.413865 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72e5f3-aeec-47d9-86ee-3ff15c9c700b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:46.414134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72e5f3-aeec-47d9-86ee-3ff15c9c700b admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:46.414749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c72e5f3-aeec-47d9-86ee-3ff15c9c700b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:46.416704 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c72e5f3-aeec-47d9-86ee-3ff15c9c700b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:46.437840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1059/2136] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:55:46 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:46.442200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797a578-4dd6-4084-a0e0-b4d288398959 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:46.452343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797a578-4dd6-4084-a0e0-b4d288398959 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:46.456954 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b7054-5800-4d25-9652-c7b3b7c86d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:46.457542 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b7054-5800-4d25-9652-c7b3b7c86d41 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:46.457816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b7054-5800-4d25-9652-c7b3b7c86d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:46.458085 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b7054-5800-4d25-9652-c7b3b7c86d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:46.458712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b7054-5800-4d25-9652-c7b3b7c86d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:46.460344 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b7054-5800-4d25-9652-c7b3b7c86d41 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:46.470271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1078/2137] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:46.474702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72e5f3-aeec-47d9-86ee-3ff15c9c700b admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:46.484922 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72e5f3-aeec-47d9-86ee-3ff15c9c700b admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:46.490820 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c6c91-482a-4fab-a67e-93787ccdcc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:46.491424 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c6c91-482a-4fab-a67e-93787ccdcc91 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:46.491740 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c6c91-482a-4fab-a67e-93787ccdcc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:46.492015 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c6c91-482a-4fab-a67e-93787ccdcc91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:46.492713 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c6c91-482a-4fab-a67e-93787ccdcc91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:46.494316 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c6c91-482a-4fab-a67e-93787ccdcc91 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:46.505416 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1060/2138] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:46.509746 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b7054-5800-4d25-9652-c7b3b7c86d41 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:46.520984 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b7054-5800-4d25-9652-c7b3b7c86d41 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:46.525628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b533e-af27-43d9-99fb-41d62eb12875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:46.526267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b533e-af27-43d9-99fb-41d62eb12875 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:46.526549 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b533e-af27-43d9-99fb-41d62eb12875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:46.526812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b533e-af27-43d9-99fb-41d62eb12875 admin admin] PATH_INFO: `/v3/projects/97feebaa044841d9a207350b346c9c8b/users/1e6e754a57c74383957fd1ab868434cc/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:46.541826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b533e-af27-43d9-99fb-41d62eb12875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97feebaa044841d9a207350b346c9c8b, user_id=1e6e754a57c74383957fd1ab868434cc, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:46.545216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b533e-af27-43d9-99fb-41d62eb12875 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:46.557786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1079/2139] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:46 2022] PUT /identity/v3/projects/97feebaa044841d9a207350b346c9c8b/users/1e6e754a57c74383957fd1ab868434cc/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:55:46.573628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f453f31-c402-4b2b-910a-33b7e7b7e848 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:46.574037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f453f31-c402-4b2b-910a-33b7e7b7e848 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:46.574488 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f453f31-c402-4b2b-910a-33b7e7b7e848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:46.604794 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-9f453f31-c402-4b2b-910a-33b7e7b7e848 None None] MFA Rules not processed for user `1e6e754a57c74383957fd1ab868434cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:46.606085 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9f453f31-c402-4b2b-910a-33b7e7b7e848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:46.673172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1061/2140] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:46.678241 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b533e-af27-43d9-99fb-41d62eb12875 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:46.725797 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:46.741262 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b533e-af27-43d9-99fb-41d62eb12875 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:46.745887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ec7e7-875a-48a6-a2f3-14fb272e4a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:46.746488 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ec7e7-875a-48a6-a2f3-14fb272e4a2c admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:46.746761 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ec7e7-875a-48a6-a2f3-14fb272e4a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:46.747028 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ec7e7-875a-48a6-a2f3-14fb272e4a2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:46.747669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ec7e7-875a-48a6-a2f3-14fb272e4a2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:46.749558 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ec7e7-875a-48a6-a2f3-14fb272e4a2c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:46.776409 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1080/2141] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:55:46 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:46.780628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f453f31-c402-4b2b-910a-33b7e7b7e848 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:46.791181 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f453f31-c402-4b2b-910a-33b7e7b7e848 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:46.848739 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6058f-718b-47a0-920a-02784d0182cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:46.849348 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6058f-718b-47a0-920a-02784d0182cf admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:46.849628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6058f-718b-47a0-920a-02784d0182cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:46.849899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6058f-718b-47a0-920a-02784d0182cf admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:46.850524 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6058f-718b-47a0-920a-02784d0182cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:46.852466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6058f-718b-47a0-920a-02784d0182cf admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:46.875406 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1062/2142] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:55:46 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:46.880113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ec7e7-875a-48a6-a2f3-14fb272e4a2c admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:46.891837 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ec7e7-875a-48a6-a2f3-14fb272e4a2c admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:46.897651 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a15070-c8ba-4b76-82cd-e73b720b75d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:46.898423 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a15070-c8ba-4b76-82cd-e73b720b75d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:46.899034 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a15070-c8ba-4b76-82cd-e73b720b75d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:46.899309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a15070-c8ba-4b76-82cd-e73b720b75d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:46.899972 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a15070-c8ba-4b76-82cd-e73b720b75d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:46.901563 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a15070-c8ba-4b76-82cd-e73b720b75d4 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:46.911992 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1081/2143] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:46.916280 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6058f-718b-47a0-920a-02784d0182cf admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:46.926865 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6058f-718b-47a0-920a-02784d0182cf admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:46.931762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded8b39-cfc2-4fb4-bfd4-47f9c2f409fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:46.932499 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded8b39-cfc2-4fb4-bfd4-47f9c2f409fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:46.932851 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded8b39-cfc2-4fb4-bfd4-47f9c2f409fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:46.933190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded8b39-cfc2-4fb4-bfd4-47f9c2f409fa admin admin] PATH_INFO: `/v3/projects/a242dc1b473f49b7937b9075c390a66d/users/89387f9755d64d83a9b422f394a566a1/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:46.948947 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded8b39-cfc2-4fb4-bfd4-47f9c2f409fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a242dc1b473f49b7937b9075c390a66d, user_id=89387f9755d64d83a9b422f394a566a1, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:46.952069 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded8b39-cfc2-4fb4-bfd4-47f9c2f409fa admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:46.965573 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1063/2144] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:46 2022] PUT /identity/v3/projects/a242dc1b473f49b7937b9075c390a66d/users/89387f9755d64d83a9b422f394a566a1/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:55:46.970511 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a15070-c8ba-4b76-82cd-e73b720b75d4 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:46.980867 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a15070-c8ba-4b76-82cd-e73b720b75d4 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:47.053586 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57c0a4-d89c-443e-a5df-6d6e177b354a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:47.054321 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57c0a4-d89c-443e-a5df-6d6e177b354a admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:47.054680 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57c0a4-d89c-443e-a5df-6d6e177b354a admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:47.055016 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57c0a4-d89c-443e-a5df-6d6e177b354a admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:47.056076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57c0a4-d89c-443e-a5df-6d6e177b354a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:47.057898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57c0a4-d89c-443e-a5df-6d6e177b354a admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:47.073670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1082/2145] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:55:47.075235 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded8b39-cfc2-4fb4-bfd4-47f9c2f409fa admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:47.086346 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded8b39-cfc2-4fb4-bfd4-47f9c2f409fa admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:47.091933 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbb96e-8d12-4109-a477-5022c4515fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:47.092597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbb96e-8d12-4109-a477-5022c4515fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:47.092877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbb96e-8d12-4109-a477-5022c4515fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:47.093151 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbb96e-8d12-4109-a477-5022c4515fe7 admin admin] PATH_INFO: `/v3/projects/a242dc1b473f49b7937b9075c390a66d/users/89387f9755d64d83a9b422f394a566a1/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:47.095135 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbb96e-8d12-4109-a477-5022c4515fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a242dc1b473f49b7937b9075c390a66d, user_id=89387f9755d64d83a9b422f394a566a1, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:47.100218 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbb96e-8d12-4109-a477-5022c4515fe7 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:47.126677 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1064/2146] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:47 2022] PUT /identity/v3/projects/a242dc1b473f49b7937b9075c390a66d/users/89387f9755d64d83a9b422f394a566a1/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:47.138713 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9ae6d-a42e-4373-aedc-6417328138a0 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:47.139140 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9ae6d-a42e-4373-aedc-6417328138a0 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:47.139535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9ae6d-a42e-4373-aedc-6417328138a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:47.181585 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-c6f9ae6d-a42e-4373-aedc-6417328138a0 None None] MFA Rules not processed for user `89387f9755d64d83a9b422f394a566a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:47.182805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-c6f9ae6d-a42e-4373-aedc-6417328138a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:47.271203 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1083/2147] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:47.281551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbb96e-8d12-4109-a477-5022c4515fe7 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:47.283315 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2cbbb96e-8d12-4109-a477-5022c4515fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:47.293590 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:47.351932 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbb96e-8d12-4109-a477-5022c4515fe7 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:47.355810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f924cf8-81ee-4e51-9a28-0175b33e2f03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:47.356447 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f924cf8-81ee-4e51-9a28-0175b33e2f03 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:47.356723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f924cf8-81ee-4e51-9a28-0175b33e2f03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:47.357002 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f924cf8-81ee-4e51-9a28-0175b33e2f03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:47.366848 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f924cf8-81ee-4e51-9a28-0175b33e2f03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:47.368283 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f924cf8-81ee-4e51-9a28-0175b33e2f03 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:47.377999 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1065/2148] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:55:47.481146 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9ae6d-a42e-4373-aedc-6417328138a0 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:47.490706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9ae6d-a42e-4373-aedc-6417328138a0 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:47.494674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408cf0a2-7749-4096-b66e-e988d110c3b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:47.495393 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cf0a2-7749-4096-b66e-e988d110c3b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:47.495706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cf0a2-7749-4096-b66e-e988d110c3b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:47.495974 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cf0a2-7749-4096-b66e-e988d110c3b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:47.543048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:47.560392 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408cf0a2-7749-4096-b66e-e988d110c3b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:47.561993 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408cf0a2-7749-4096-b66e-e988d110c3b4 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:47.572287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1084/2149] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:55:49.408530 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f924cf8-81ee-4e51-9a28-0175b33e2f03 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:49.409949 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8f924cf8-81ee-4e51-9a28-0175b33e2f03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:49.468076 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f924cf8-81ee-4e51-9a28-0175b33e2f03 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:49.471560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bb927-83fb-4c5e-875d-d52f0e30ce01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:49.472178 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bb927-83fb-4c5e-875d-d52f0e30ce01 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:49.472528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bb927-83fb-4c5e-875d-d52f0e30ce01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:49.472729 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bb927-83fb-4c5e-875d-d52f0e30ce01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:49.474551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-756bb927-83fb-4c5e-875d-d52f0e30ce01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:49.535112 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756bb927-83fb-4c5e-875d-d52f0e30ce01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:49.536548 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756bb927-83fb-4c5e-875d-d52f0e30ce01 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:49.545880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1066/2150] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:55:50.195899 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408cf0a2-7749-4096-b66e-e988d110c3b4 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:50.198923 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-408cf0a2-7749-4096-b66e-e988d110c3b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:50.214257 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408cf0a2-7749-4096-b66e-e988d110c3b4 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:50.221646 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86d034-a1d7-4a6a-a07e-235534905913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:50.222578 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86d034-a1d7-4a6a-a07e-235534905913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:50.223058 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86d034-a1d7-4a6a-a07e-235534905913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:50.224336 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86d034-a1d7-4a6a-a07e-235534905913 admin admin] PATH_INFO: `/v3/users/8ecc2579ccf9406383a6eb3a0b9356cb` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:50.240106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86d034-a1d7-4a6a-a07e-235534905913 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ecc2579ccf9406383a6eb3a0b9356cb)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:50.242240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86d034-a1d7-4a6a-a07e-235534905913 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:50.307541 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ee86d034-a1d7-4a6a-a07e-235534905913 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ecc2579ccf9406383a6eb3a0b9356cb'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:50.320786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ee86d034-a1d7-4a6a-a07e-235534905913 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ecc2579ccf9406383a6eb3a0b9356cb'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:50.327780 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ee86d034-a1d7-4a6a-a07e-235534905913 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ecc2579ccf9406383a6eb3a0b9356cb'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:50.328404 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ee86d034-a1d7-4a6a-a07e-235534905913 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ecc2579ccf9406383a6eb3a0b9356cb'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:50.336097 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ee86d034-a1d7-4a6a-a07e-235534905913 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ecc2579ccf9406383a6eb3a0b9356cb'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:50.342133 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1085/2151] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:55:50 2022] DELETE /identity/v3/users/8ecc2579ccf9406383a6eb3a0b9356cb => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:50.522297 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bb927-83fb-4c5e-875d-d52f0e30ce01 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:50.532540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bb927-83fb-4c5e-875d-d52f0e30ce01 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:50.611049 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df32f16-ee9c-4215-b535-8d502d0f2d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:50.612332 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df32f16-ee9c-4215-b535-8d502d0f2d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:50.612676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df32f16-ee9c-4215-b535-8d502d0f2d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:50.613010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df32f16-ee9c-4215-b535-8d502d0f2d1f admin admin] PATH_INFO: `/v3/projects/dcfe635652d548098c058c5fbf1319c2` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:50.615073 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:50.625200 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df32f16-ee9c-4215-b535-8d502d0f2d1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcfe635652d548098c058c5fbf1319c2)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:50.627246 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df32f16-ee9c-4215-b535-8d502d0f2d1f admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:50.687194 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0df32f16-ee9c-4215-b535-8d502d0f2d1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcfe635652d548098c058c5fbf1319c2', 'request_id': 'req-0df32f16-ee9c-4215-b535-8d502d0f2d1f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:50.697663 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0df32f16-ee9c-4215-b535-8d502d0f2d1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcfe635652d548098c058c5fbf1319c2', 'request_id': 'req-0df32f16-ee9c-4215-b535-8d502d0f2d1f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:50.703662 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0df32f16-ee9c-4215-b535-8d502d0f2d1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcfe635652d548098c058c5fbf1319c2', 'request_id': 'req-0df32f16-ee9c-4215-b535-8d502d0f2d1f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:50.715360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0df32f16-ee9c-4215-b535-8d502d0f2d1f admin admin] The token cache is being invalidate because project dcfe635652d548098c058c5fbf1319c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:55:50.715806 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0df32f16-ee9c-4215-b535-8d502d0f2d1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:50.716456 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0df32f16-ee9c-4215-b535-8d502d0f2d1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:50.718352 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1067/2152] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:55:50 2022] DELETE /identity/v3/projects/dcfe635652d548098c058c5fbf1319c2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:55:50.724285 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2519f-1d4c-4906-896e-bc4116505ffb None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:50.724637 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2519f-1d4c-4906-896e-bc4116505ffb None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:50.725062 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2519f-1d4c-4906-896e-bc4116505ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:50.751701 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-52a2519f-1d4c-4906-896e-bc4116505ffb None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:50.753492 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-52a2519f-1d4c-4906-896e-bc4116505ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:50.816811 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1086/2153] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:50.825943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1033ea-e2f6-4f38-b564-b2063adf5204 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:50.826325 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1033ea-e2f6-4f38-b564-b2063adf5204 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:50.826665 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1033ea-e2f6-4f38-b564-b2063adf5204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:50.847830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-0a1033ea-e2f6-4f38-b564-b2063adf5204 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:50.849136 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-0a1033ea-e2f6-4f38-b564-b2063adf5204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:50.857308 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1068/2154] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:50.861637 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2519f-1d4c-4906-896e-bc4116505ffb None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:50.872038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2519f-1d4c-4906-896e-bc4116505ffb None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:50.877244 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bec01d-3da5-4a76-8d2c-2e94a66534b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:50.877843 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bec01d-3da5-4a76-8d2c-2e94a66534b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:50.878119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bec01d-3da5-4a76-8d2c-2e94a66534b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:50.878387 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bec01d-3da5-4a76-8d2c-2e94a66534b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:50.879023 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bec01d-3da5-4a76-8d2c-2e94a66534b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:55:50.879323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bec01d-3da5-4a76-8d2c-2e94a66534b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:50.880950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bec01d-3da5-4a76-8d2c-2e94a66534b9 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:50.897149 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1087/2155] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:55:50.904590 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1033ea-e2f6-4f38-b564-b2063adf5204 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:50.914340 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1033ea-e2f6-4f38-b564-b2063adf5204 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:50.919024 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f38e36-3a5b-4d85-b434-01d640363369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:50.919672 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f38e36-3a5b-4d85-b434-01d640363369 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:50.919945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f38e36-3a5b-4d85-b434-01d640363369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:50.920213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f38e36-3a5b-4d85-b434-01d640363369 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:50.920848 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f38e36-3a5b-4d85-b434-01d640363369 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:50.922769 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f38e36-3a5b-4d85-b434-01d640363369 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:50.955787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1069/2156] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:55:50 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:50.958074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bec01d-3da5-4a76-8d2c-2e94a66534b9 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:50.968164 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bec01d-3da5-4a76-8d2c-2e94a66534b9 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:51.004230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:51.042135 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b0ec8-93bd-4baf-88a6-36ba56bb0ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:51.042135 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b0ec8-93bd-4baf-88a6-36ba56bb0ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:51.042135 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b0ec8-93bd-4baf-88a6-36ba56bb0ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:51.042135 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b0ec8-93bd-4baf-88a6-36ba56bb0ab6 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:51.042801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b0ec8-93bd-4baf-88a6-36ba56bb0ab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:51.044679 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b0ec8-93bd-4baf-88a6-36ba56bb0ab6 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:51.076869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1088/2157] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:55:50 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:51.078440 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f38e36-3a5b-4d85-b434-01d640363369 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:51.098170 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f38e36-3a5b-4d85-b434-01d640363369 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:51.104385 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b812d-66fe-4c0a-b099-8cb6a7a38085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:51.105171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b812d-66fe-4c0a-b099-8cb6a7a38085 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:51.105630 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b812d-66fe-4c0a-b099-8cb6a7a38085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:51.106073 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b812d-66fe-4c0a-b099-8cb6a7a38085 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:51.106862 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b812d-66fe-4c0a-b099-8cb6a7a38085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:51.108625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b812d-66fe-4c0a-b099-8cb6a7a38085 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:51.121125 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1070/2158] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:51.125196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b0ec8-93bd-4baf-88a6-36ba56bb0ab6 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:51.137622 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b0ec8-93bd-4baf-88a6-36ba56bb0ab6 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:51.142659 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b9fe8-7446-446a-ace9-f4c662424bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:51.143319 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b9fe8-7446-446a-ace9-f4c662424bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:51.143718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b9fe8-7446-446a-ace9-f4c662424bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:51.143957 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b9fe8-7446-446a-ace9-f4c662424bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:51.144601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b9fe8-7446-446a-ace9-f4c662424bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:51.146274 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b9fe8-7446-446a-ace9-f4c662424bc2 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:51.161656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1089/2159] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:51.166422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b812d-66fe-4c0a-b099-8cb6a7a38085 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:51.181443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b812d-66fe-4c0a-b099-8cb6a7a38085 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:51.186993 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e215c63-ffb2-4f11-816d-7281bfbd88e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:51.187814 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e215c63-ffb2-4f11-816d-7281bfbd88e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:51.188197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e215c63-ffb2-4f11-816d-7281bfbd88e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:51.188545 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e215c63-ffb2-4f11-816d-7281bfbd88e8 admin admin] PATH_INFO: `/v3/projects/06e73028dc934238b13f87ed0719038e/users/933f34b82d5b4355844a57707de417a1/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:51.203290 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b9fe8-7446-446a-ace9-f4c662424bc2 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:51.207235 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-6c3b9fe8-7446-446a-ace9-f4c662424bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:51.211011 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e215c63-ffb2-4f11-816d-7281bfbd88e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e73028dc934238b13f87ed0719038e, user_id=933f34b82d5b4355844a57707de417a1, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:51.213720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e215c63-ffb2-4f11-816d-7281bfbd88e8 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:51.225642 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b9fe8-7446-446a-ace9-f4c662424bc2 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:51.233933 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1071/2160] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:51 2022] PUT /identity/v3/projects/06e73028dc934238b13f87ed0719038e/users/933f34b82d5b4355844a57707de417a1/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:51.238619 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a41d75-ad83-4c50-a968-3e84cd0661a9 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:51.239077 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a41d75-ad83-4c50-a968-3e84cd0661a9 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:51.239362 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a41d75-ad83-4c50-a968-3e84cd0661a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:51.261710 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-19a41d75-ad83-4c50-a968-3e84cd0661a9 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:51.262986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-19a41d75-ad83-4c50-a968-3e84cd0661a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:51.289095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26907497-525e-41b6-bdc0-ca8823c1a113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:51.289730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26907497-525e-41b6-bdc0-ca8823c1a113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:51.290002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26907497-525e-41b6-bdc0-ca8823c1a113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:51.290272 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26907497-525e-41b6-bdc0-ca8823c1a113 admin admin] PATH_INFO: `/v3/users/3cf3a49826ac480b9737068165545e0f` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:51.291413 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26907497-525e-41b6-bdc0-ca8823c1a113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cf3a49826ac480b9737068165545e0f)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:51.293563 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26907497-525e-41b6-bdc0-ca8823c1a113 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:51.337402 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1072/2161] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:55:51.348437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a41d75-ad83-4c50-a968-3e84cd0661a9 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:51.352225 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-19a41d75-ad83-4c50-a968-3e84cd0661a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:51.361571 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-26907497-525e-41b6-bdc0-ca8823c1a113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cf3a49826ac480b9737068165545e0f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:51.373842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-26907497-525e-41b6-bdc0-ca8823c1a113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cf3a49826ac480b9737068165545e0f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:51.375024 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:51.381583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-26907497-525e-41b6-bdc0-ca8823c1a113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cf3a49826ac480b9737068165545e0f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:51.382510 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-26907497-525e-41b6-bdc0-ca8823c1a113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cf3a49826ac480b9737068165545e0f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:51.394223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-26907497-525e-41b6-bdc0-ca8823c1a113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cf3a49826ac480b9737068165545e0f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:51.401326 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1090/2162] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:55:51 2022] DELETE /identity/v3/users/3cf3a49826ac480b9737068165545e0f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:51.405222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26907497-525e-41b6-bdc0-ca8823c1a113 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:51.406754 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-26907497-525e-41b6-bdc0-ca8823c1a113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:51.420638 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a41d75-ad83-4c50-a968-3e84cd0661a9 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:51.449522 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:51.487984 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065020d-c514-4a34-8009-e78d080e21f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:51.488985 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065020d-c514-4a34-8009-e78d080e21f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:51.489365 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065020d-c514-4a34-8009-e78d080e21f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:51.489737 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065020d-c514-4a34-8009-e78d080e21f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:51.492192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8065020d-c514-4a34-8009-e78d080e21f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:51.501823 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26907497-525e-41b6-bdc0-ca8823c1a113 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:51.506182 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8065020d-c514-4a34-8009-e78d080e21f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:51.507784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8065020d-c514-4a34-8009-e78d080e21f8 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:51.508769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aaee15-5c8e-4304-9b75-c59fe3bea683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:51.509471 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aaee15-5c8e-4304-9b75-c59fe3bea683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:51.509754 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aaee15-5c8e-4304-9b75-c59fe3bea683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:51.510909 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aaee15-5c8e-4304-9b75-c59fe3bea683 admin admin] PATH_INFO: `/v3/users/1e6e754a57c74383957fd1ab868434cc` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:51.512160 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aaee15-5c8e-4304-9b75-c59fe3bea683 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e6e754a57c74383957fd1ab868434cc)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:51.514915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aaee15-5c8e-4304-9b75-c59fe3bea683 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:51.520779 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1073/2163] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:55:51.571735 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d9aaee15-5c8e-4304-9b75-c59fe3bea683 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e6e754a57c74383957fd1ab868434cc'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:51.587591 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d9aaee15-5c8e-4304-9b75-c59fe3bea683 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e6e754a57c74383957fd1ab868434cc'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:51.597304 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d9aaee15-5c8e-4304-9b75-c59fe3bea683 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e6e754a57c74383957fd1ab868434cc'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:51.598295 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d9aaee15-5c8e-4304-9b75-c59fe3bea683 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e6e754a57c74383957fd1ab868434cc'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:51.609825 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-d9aaee15-5c8e-4304-9b75-c59fe3bea683 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e6e754a57c74383957fd1ab868434cc'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:51.617049 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1091/2164] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:55:51 2022] DELETE /identity/v3/users/1e6e754a57c74383957fd1ab868434cc => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:51.621577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d545276-a0d3-480f-b0ed-fa4a5e285bd8 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:51.622110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d545276-a0d3-480f-b0ed-fa4a5e285bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:51.622236 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d545276-a0d3-480f-b0ed-fa4a5e285bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:51.647615 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-9d545276-a0d3-480f-b0ed-fa4a5e285bd8 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:51.648869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9d545276-a0d3-480f-b0ed-fa4a5e285bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:51.655225 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aaee15-5c8e-4304-9b75-c59fe3bea683 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:51.656695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d9aaee15-5c8e-4304-9b75-c59fe3bea683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:51.747863 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aaee15-5c8e-4304-9b75-c59fe3bea683 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:51.751728 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1074/2165] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:51.754565 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:51.755613 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:51.755613 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:51.756606 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5 admin admin] PATH_INFO: `/v3/projects/69bb5ae23eb1436c915d0e3407012255` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:51.757464 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69bb5ae23eb1436c915d0e3407012255)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:51.760594 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:51.770632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d545276-a0d3-480f-b0ed-fa4a5e285bd8 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:51.773534 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9d545276-a0d3-480f-b0ed-fa4a5e285bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:51.853879 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69bb5ae23eb1436c915d0e3407012255', 'request_id': 'req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:51.855048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:51.863740 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69bb5ae23eb1436c915d0e3407012255', 'request_id': 'req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:51.873855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d545276-a0d3-480f-b0ed-fa4a5e285bd8 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:51.876049 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69bb5ae23eb1436c915d0e3407012255', 'request_id': 'req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:51.894952 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5 admin admin] The token cache is being invalidate because project 69bb5ae23eb1436c915d0e3407012255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:55:51.895510 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:51.896154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:51.898611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1092/2166] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:55:51 2022] DELETE /identity/v3/projects/69bb5ae23eb1436c915d0e3407012255 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:55:51.902468 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:51.904430 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:51.926379 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:51.958770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a64ed0-8e0e-42c8-bb61-20998f785596 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:51.959451 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a64ed0-8e0e-42c8-bb61-20998f785596 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:51.959765 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a64ed0-8e0e-42c8-bb61-20998f785596 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:51.960041 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a64ed0-8e0e-42c8-bb61-20998f785596 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:51.965835 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a6a64ed0-8e0e-42c8-bb61-20998f785596 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:51.998839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867330f-ad74-4a3a-89df-ddc9d5fb30b5 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:52.006747 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c0e78-91b4-44cb-b853-666b8d29af5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:52.008025 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c0e78-91b4-44cb-b853-666b8d29af5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:52.008405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c0e78-91b4-44cb-b853-666b8d29af5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:52.008847 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c0e78-91b4-44cb-b853-666b8d29af5c admin admin] PATH_INFO: `/v3/users/98fc5376ab1444a597fbfc5c2da8ed37` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:52.010765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c0e78-91b4-44cb-b853-666b8d29af5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=98fc5376ab1444a597fbfc5c2da8ed37)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:52.013314 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c0e78-91b4-44cb-b853-666b8d29af5c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:52.060934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a64ed0-8e0e-42c8-bb61-20998f785596 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:52.062719 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a64ed0-8e0e-42c8-bb61-20998f785596 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:52.078664 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1075/2167] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:55:52.100094 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-730c0e78-91b4-44cb-b853-666b8d29af5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98fc5376ab1444a597fbfc5c2da8ed37'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:52.125473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-730c0e78-91b4-44cb-b853-666b8d29af5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98fc5376ab1444a597fbfc5c2da8ed37'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:52.144028 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-730c0e78-91b4-44cb-b853-666b8d29af5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98fc5376ab1444a597fbfc5c2da8ed37'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:52.145210 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-730c0e78-91b4-44cb-b853-666b8d29af5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98fc5376ab1444a597fbfc5c2da8ed37'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:52.157692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-730c0e78-91b4-44cb-b853-666b8d29af5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98fc5376ab1444a597fbfc5c2da8ed37'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:52.172057 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1093/2168] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:55:51 2022] DELETE /identity/v3/users/98fc5376ab1444a597fbfc5c2da8ed37 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:55:52.566634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a64ed0-8e0e-42c8-bb61-20998f785596 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:52.568379 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a6a64ed0-8e0e-42c8-bb61-20998f785596 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:52.585079 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a64ed0-8e0e-42c8-bb61-20998f785596 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:52.595963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c0e78-91b4-44cb-b853-666b8d29af5c admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:52.597778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-730c0e78-91b4-44cb-b853-666b8d29af5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:52.729597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f8875-1b01-4499-99c8-c2576bb695a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:52.730348 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f8875-1b01-4499-99c8-c2576bb695a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:52.730619 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f8875-1b01-4499-99c8-c2576bb695a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:52.730896 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f8875-1b01-4499-99c8-c2576bb695a2 admin admin] PATH_INFO: `/v3/projects/97feebaa044841d9a207350b346c9c8b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:52.734932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:52.737966 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f8875-1b01-4499-99c8-c2576bb695a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97feebaa044841d9a207350b346c9c8b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:52.739956 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f8875-1b01-4499-99c8-c2576bb695a2 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:52.743880 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c0e78-91b4-44cb-b853-666b8d29af5c admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:52.753650 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fac993-9b5c-4962-be68-adb9c7424620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:52.755488 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fac993-9b5c-4962-be68-adb9c7424620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:52.756026 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fac993-9b5c-4962-be68-adb9c7424620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:52.756520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fac993-9b5c-4962-be68-adb9c7424620 admin admin] PATH_INFO: `/v3/projects/c7c2f47dede54bf091d9e875ba974b4b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:52.757991 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fac993-9b5c-4962-be68-adb9c7424620 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7c2f47dede54bf091d9e875ba974b4b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:52.761198 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fac993-9b5c-4962-be68-adb9c7424620 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:52.827323 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-531f8875-1b01-4499-99c8-c2576bb695a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97feebaa044841d9a207350b346c9c8b', 'request_id': 'req-531f8875-1b01-4499-99c8-c2576bb695a2'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:52.837899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-531f8875-1b01-4499-99c8-c2576bb695a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97feebaa044841d9a207350b346c9c8b', 'request_id': 'req-531f8875-1b01-4499-99c8-c2576bb695a2'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:52.849516 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-22fac993-9b5c-4962-be68-adb9c7424620 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7c2f47dede54bf091d9e875ba974b4b', 'request_id': 'req-22fac993-9b5c-4962-be68-adb9c7424620'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:52.850332 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-531f8875-1b01-4499-99c8-c2576bb695a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97feebaa044841d9a207350b346c9c8b', 'request_id': 'req-531f8875-1b01-4499-99c8-c2576bb695a2'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:52.858165 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-22fac993-9b5c-4962-be68-adb9c7424620 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7c2f47dede54bf091d9e875ba974b4b', 'request_id': 'req-22fac993-9b5c-4962-be68-adb9c7424620'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:52.869210 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-531f8875-1b01-4499-99c8-c2576bb695a2 admin admin] The token cache is being invalidate because project 97feebaa044841d9a207350b346c9c8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:55:52.870258 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-531f8875-1b01-4499-99c8-c2576bb695a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:52.871348 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-531f8875-1b01-4499-99c8-c2576bb695a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:52.873814 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1076/2169] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:55:52 2022] DELETE /identity/v3/projects/97feebaa044841d9a207350b346c9c8b => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:52.879468 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f8875-1b01-4499-99c8-c2576bb695a2 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:52.880504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-22fac993-9b5c-4962-be68-adb9c7424620 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7c2f47dede54bf091d9e875ba974b4b', 'request_id': 'req-22fac993-9b5c-4962-be68-adb9c7424620'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:52.882878 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-531f8875-1b01-4499-99c8-c2576bb695a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:52.902333 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-22fac993-9b5c-4962-be68-adb9c7424620 admin admin] The token cache is being invalidate because project c7c2f47dede54bf091d9e875ba974b4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:55:52.902632 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-22fac993-9b5c-4962-be68-adb9c7424620 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:52.903640 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-22fac993-9b5c-4962-be68-adb9c7424620 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:52.905315 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1094/2170] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:55:52 2022] DELETE /identity/v3/projects/c7c2f47dede54bf091d9e875ba974b4b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:55:52.909781 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaeda7-5dea-43a8-b32f-0c94195e0ae9 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:52.911725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaeda7-5dea-43a8-b32f-0c94195e0ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:52.912258 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaeda7-5dea-43a8-b32f-0c94195e0ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:52.918453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:52.942695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-facaeda7-5dea-43a8-b32f-0c94195e0ae9 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:52.944305 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-facaeda7-5dea-43a8-b32f-0c94195e0ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:52.999046 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f8875-1b01-4499-99c8-c2576bb695a2 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:53.040269 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1095/2171] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:55:53.050663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16185f0b-5591-444a-8355-53b0918e1c37 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:53.051007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16185f0b-5591-444a-8355-53b0918e1c37 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:53.051292 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16185f0b-5591-444a-8355-53b0918e1c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:53.092611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-16185f0b-5591-444a-8355-53b0918e1c37 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:53.094558 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-16185f0b-5591-444a-8355-53b0918e1c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:53.106533 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf17d9b-c7cc-4a19-a39b-ea40d5fcc3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:53.107239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf17d9b-c7cc-4a19-a39b-ea40d5fcc3b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:53.107525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf17d9b-c7cc-4a19-a39b-ea40d5fcc3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:53.107836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf17d9b-c7cc-4a19-a39b-ea40d5fcc3b9 admin admin] PATH_INFO: `/v3/users/89387f9755d64d83a9b422f394a566a1` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:53.109047 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf17d9b-c7cc-4a19-a39b-ea40d5fcc3b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89387f9755d64d83a9b422f394a566a1)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:53.111175 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf17d9b-c7cc-4a19-a39b-ea40d5fcc3b9 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:53.117058 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1096/2172] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:53.120929 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16185f0b-5591-444a-8355-53b0918e1c37 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:53.136278 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16185f0b-5591-444a-8355-53b0918e1c37 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:53.144093 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33f90d-a425-41c1-9dcd-b0ba9bdffdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:53.144825 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33f90d-a425-41c1-9dcd-b0ba9bdffdd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:53.145103 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33f90d-a425-41c1-9dcd-b0ba9bdffdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:53.145371 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33f90d-a425-41c1-9dcd-b0ba9bdffdd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:53.146267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce33f90d-a425-41c1-9dcd-b0ba9bdffdd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:55:53.146539 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce33f90d-a425-41c1-9dcd-b0ba9bdffdd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:53.148828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce33f90d-a425-41c1-9dcd-b0ba9bdffdd5 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:53.165256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1097/2173] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:55:53.169681 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33f90d-a425-41c1-9dcd-b0ba9bdffdd5 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:53.181664 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9bf17d9b-c7cc-4a19-a39b-ea40d5fcc3b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89387f9755d64d83a9b422f394a566a1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:53.182672 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9bf17d9b-c7cc-4a19-a39b-ea40d5fcc3b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89387f9755d64d83a9b422f394a566a1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:53.216877 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33f90d-a425-41c1-9dcd-b0ba9bdffdd5 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:53.217847 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9bf17d9b-c7cc-4a19-a39b-ea40d5fcc3b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89387f9755d64d83a9b422f394a566a1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:53.228233 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32adcce7-d7ca-4aa7-93f1-68cee768ae28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:53.229387 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adcce7-d7ca-4aa7-93f1-68cee768ae28 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:53.229876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adcce7-d7ca-4aa7-93f1-68cee768ae28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:53.230501 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adcce7-d7ca-4aa7-93f1-68cee768ae28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:53.231432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32adcce7-d7ca-4aa7-93f1-68cee768ae28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:53.234117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32adcce7-d7ca-4aa7-93f1-68cee768ae28 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:53.241361 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9bf17d9b-c7cc-4a19-a39b-ea40d5fcc3b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89387f9755d64d83a9b422f394a566a1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:53.244379 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9bf17d9b-c7cc-4a19-a39b-ea40d5fcc3b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89387f9755d64d83a9b422f394a566a1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:53.287617 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1077/2174] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:55:52 2022] DELETE /identity/v3/users/89387f9755d64d83a9b422f394a566a1 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:53.290625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf17d9b-c7cc-4a19-a39b-ea40d5fcc3b9 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:53.292910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9bf17d9b-c7cc-4a19-a39b-ea40d5fcc3b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:53.302928 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1098/2175] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:55:53 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:53.405449 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:53.450342 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf17d9b-c7cc-4a19-a39b-ea40d5fcc3b9 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:53.465740 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814219a9-32d0-4e73-8928-35bfdbe4c6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:53.466475 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814219a9-32d0-4e73-8928-35bfdbe4c6be admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:53.467238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814219a9-32d0-4e73-8928-35bfdbe4c6be admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:53.467238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814219a9-32d0-4e73-8928-35bfdbe4c6be admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:53.467951 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814219a9-32d0-4e73-8928-35bfdbe4c6be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:53.469965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814219a9-32d0-4e73-8928-35bfdbe4c6be admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:53.524892 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1078/2176] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:55:53 2022] POST /identity/v3/users => generated 286 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:53.526166 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32adcce7-d7ca-4aa7-93f1-68cee768ae28 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:53.614001 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:53.640180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32adcce7-d7ca-4aa7-93f1-68cee768ae28 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:53.647337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3b725-b99f-4e8c-b89d-9f652eeaa7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:53.647774 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3b725-b99f-4e8c-b89d-9f652eeaa7da admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:53.648100 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3b725-b99f-4e8c-b89d-9f652eeaa7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:53.649622 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3b725-b99f-4e8c-b89d-9f652eeaa7da admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:53.650292 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc3b725-b99f-4e8c-b89d-9f652eeaa7da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:53.653768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc3b725-b99f-4e8c-b89d-9f652eeaa7da admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:53.671819 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1099/2177] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:53.675841 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814219a9-32d0-4e73-8928-35bfdbe4c6be admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:53.692849 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814219a9-32d0-4e73-8928-35bfdbe4c6be admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:53.699386 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217618d-779f-4911-9513-97e27e87f31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:53.699974 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217618d-779f-4911-9513-97e27e87f31f admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:53.700793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217618d-779f-4911-9513-97e27e87f31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:53.701243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217618d-779f-4911-9513-97e27e87f31f admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:53.702981 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217618d-779f-4911-9513-97e27e87f31f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:53.705869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217618d-779f-4911-9513-97e27e87f31f admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:53.728788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3b725-b99f-4e8c-b89d-9f652eeaa7da admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:53.729724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1079/2178] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:55:53.743971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3b725-b99f-4e8c-b89d-9f652eeaa7da admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:53.772451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e6539-48ae-423c-a912-8a4b372083d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:53.773485 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6539-48ae-423c-a912-8a4b372083d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:53.775494 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6539-48ae-423c-a912-8a4b372083d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:53.775864 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6539-48ae-423c-a912-8a4b372083d7 admin admin] PATH_INFO: `/v3/projects/4b0c6a35b8964637b86329ab2a75e95e/users/cec1835cdf354085a0828d956a2d1ec2/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:53.825233 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e6539-48ae-423c-a912-8a4b372083d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b0c6a35b8964637b86329ab2a75e95e, user_id=cec1835cdf354085a0828d956a2d1ec2, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:53.828388 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e6539-48ae-423c-a912-8a4b372083d7 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:53.858979 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1100/2179] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:53 2022] PUT /identity/v3/projects/4b0c6a35b8964637b86329ab2a75e95e/users/cec1835cdf354085a0828d956a2d1ec2/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:53.872709 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101034ba-8c1c-47c0-b4c2-03bfbe811c84 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:53.872915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101034ba-8c1c-47c0-b4c2-03bfbe811c84 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:53.873173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101034ba-8c1c-47c0-b4c2-03bfbe811c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:53.909665 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-101034ba-8c1c-47c0-b4c2-03bfbe811c84 None None] MFA Rules not processed for user `cec1835cdf354085a0828d956a2d1ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:53.912259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-101034ba-8c1c-47c0-b4c2-03bfbe811c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:54.028785 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1080/2180] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:54.037113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e6539-48ae-423c-a912-8a4b372083d7 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:54.043815 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-181e6539-48ae-423c-a912-8a4b372083d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:54.105908 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101034ba-8c1c-47c0-b4c2-03bfbe811c84 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:54.107397 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-101034ba-8c1c-47c0-b4c2-03bfbe811c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:54.174967 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e6539-48ae-423c-a912-8a4b372083d7 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:54.185284 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f260f-8f96-47ce-86d0-b934a5dc5a23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:54.186351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f260f-8f96-47ce-86d0-b934a5dc5a23 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:54.187355 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f260f-8f96-47ce-86d0-b934a5dc5a23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:54.187837 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f260f-8f96-47ce-86d0-b934a5dc5a23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:54.210731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f260f-8f96-47ce-86d0-b934a5dc5a23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:54.212234 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f260f-8f96-47ce-86d0-b934a5dc5a23 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:54.234580 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1101/2181] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:55:54.246690 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101034ba-8c1c-47c0-b4c2-03bfbe811c84 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:54.248531 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f260f-8f96-47ce-86d0-b934a5dc5a23 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:54.252109 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44404cd8-bbfd-4f0c-88d9-afa446c68f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:54.252898 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44404cd8-bbfd-4f0c-88d9-afa446c68f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:54.253280 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44404cd8-bbfd-4f0c-88d9-afa446c68f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:54.257875 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44404cd8-bbfd-4f0c-88d9-afa446c68f5e admin admin] PATH_INFO: `/v3/projects/a242dc1b473f49b7937b9075c390a66d` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:54.259389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44404cd8-bbfd-4f0c-88d9-afa446c68f5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a242dc1b473f49b7937b9075c390a66d)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:54.261701 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44404cd8-bbfd-4f0c-88d9-afa446c68f5e admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:54.289370 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f260f-8f96-47ce-86d0-b934a5dc5a23 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:54.320059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b5f9e-f9fa-4bd7-9507-cd02dd04f86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:54.321005 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b5f9e-f9fa-4bd7-9507-cd02dd04f86a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:54.321286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b5f9e-f9fa-4bd7-9507-cd02dd04f86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:54.321560 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b5f9e-f9fa-4bd7-9507-cd02dd04f86a admin admin] PATH_INFO: `/v3/users/cec1835cdf354085a0828d956a2d1ec2` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:54.323715 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b5f9e-f9fa-4bd7-9507-cd02dd04f86a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cec1835cdf354085a0828d956a2d1ec2)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:54.325643 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b5f9e-f9fa-4bd7-9507-cd02dd04f86a admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:54.356875 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-44404cd8-bbfd-4f0c-88d9-afa446c68f5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a242dc1b473f49b7937b9075c390a66d', 'request_id': 'req-44404cd8-bbfd-4f0c-88d9-afa446c68f5e'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:54.361911 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:54.379327 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-44404cd8-bbfd-4f0c-88d9-afa446c68f5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a242dc1b473f49b7937b9075c390a66d', 'request_id': 'req-44404cd8-bbfd-4f0c-88d9-afa446c68f5e'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:54.393872 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-44404cd8-bbfd-4f0c-88d9-afa446c68f5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a242dc1b473f49b7937b9075c390a66d', 'request_id': 'req-44404cd8-bbfd-4f0c-88d9-afa446c68f5e'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:54.417305 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-44404cd8-bbfd-4f0c-88d9-afa446c68f5e admin admin] The token cache is being invalidate because project a242dc1b473f49b7937b9075c390a66d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:55:54.417305 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-44404cd8-bbfd-4f0c-88d9-afa446c68f5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:54.417810 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-884b5f9e-f9fa-4bd7-9507-cd02dd04f86a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cec1835cdf354085a0828d956a2d1ec2'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:54.420163 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-44404cd8-bbfd-4f0c-88d9-afa446c68f5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:54.422177 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1081/2182] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:55:54 2022] DELETE /identity/v3/projects/a242dc1b473f49b7937b9075c390a66d => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:55:54.426790 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72269d64-770c-4bf7-968b-b78118927346 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:54.427886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72269d64-770c-4bf7-968b-b78118927346 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:54.427886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72269d64-770c-4bf7-968b-b78118927346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:54.452497 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-884b5f9e-f9fa-4bd7-9507-cd02dd04f86a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cec1835cdf354085a0828d956a2d1ec2'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:54.457963 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-72269d64-770c-4bf7-968b-b78118927346 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:54.459643 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-72269d64-770c-4bf7-968b-b78118927346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:54.469382 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-884b5f9e-f9fa-4bd7-9507-cd02dd04f86a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cec1835cdf354085a0828d956a2d1ec2'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:54.472694 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-884b5f9e-f9fa-4bd7-9507-cd02dd04f86a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cec1835cdf354085a0828d956a2d1ec2'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:54.482429 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-884b5f9e-f9fa-4bd7-9507-cd02dd04f86a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cec1835cdf354085a0828d956a2d1ec2'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:54.489805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1102/2183] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:55:54 2022] DELETE /identity/v3/users/cec1835cdf354085a0828d956a2d1ec2 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:55:54.494315 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522d2b9-78a9-4c55-baaf-2bc7145608e7 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:54.495526 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522d2b9-78a9-4c55-baaf-2bc7145608e7 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:54.498553 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522d2b9-78a9-4c55-baaf-2bc7145608e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:54.540180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-a522d2b9-78a9-4c55-baaf-2bc7145608e7 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:54.542173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-a522d2b9-78a9-4c55-baaf-2bc7145608e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:54.607503 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:54.612076 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1082/2184] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:55:54.621739 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096be7e-1323-4c97-988f-083bea881c0d None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:54.622286 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096be7e-1323-4c97-988f-083bea881c0d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:54.622286 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096be7e-1323-4c97-988f-083bea881c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:54.680770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1103/2185] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:54.681490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-f096be7e-1323-4c97-988f-083bea881c0d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:54.682822 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-f096be7e-1323-4c97-988f-083bea881c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:54.687661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522d2b9-78a9-4c55-baaf-2bc7145608e7 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:54.692903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-a522d2b9-78a9-4c55-baaf-2bc7145608e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:54.703378 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1083/2186] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:54.710515 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096be7e-1323-4c97-988f-083bea881c0d None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:54.726752 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096be7e-1323-4c97-988f-083bea881c0d None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:54.732868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb7ef3-8411-432b-a5ca-3ab0dc83b08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:54.734054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb7ef3-8411-432b-a5ca-3ab0dc83b08d admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:54.734454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb7ef3-8411-432b-a5ca-3ab0dc83b08d admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:54.734970 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb7ef3-8411-432b-a5ca-3ab0dc83b08d admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:54.738671 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb7ef3-8411-432b-a5ca-3ab0dc83b08d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:55:54.738671 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb7ef3-8411-432b-a5ca-3ab0dc83b08d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:54.740529 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb7ef3-8411-432b-a5ca-3ab0dc83b08d admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:54.776871 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1084/2187] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:55:54.777102 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb7ef3-8411-432b-a5ca-3ab0dc83b08d admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:54.788685 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb7ef3-8411-432b-a5ca-3ab0dc83b08d admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:54.794675 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac786a8f-33c5-4af8-b378-9c59fa3a6dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:54.795300 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac786a8f-33c5-4af8-b378-9c59fa3a6dfc admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:54.795767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac786a8f-33c5-4af8-b378-9c59fa3a6dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:54.796031 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac786a8f-33c5-4af8-b378-9c59fa3a6dfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:54.796869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac786a8f-33c5-4af8-b378-9c59fa3a6dfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:54.798855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac786a8f-33c5-4af8-b378-9c59fa3a6dfc admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:54.802389 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522d2b9-78a9-4c55-baaf-2bc7145608e7 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:54.806704 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc4e8d-3961-42e2-b8f0-c335d0d2570d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:54.807444 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc4e8d-3961-42e2-b8f0-c335d0d2570d service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:54.807801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc4e8d-3961-42e2-b8f0-c335d0d2570d service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:54.808240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc4e8d-3961-42e2-b8f0-c335d0d2570d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:54.819010 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc4e8d-3961-42e2-b8f0-c335d0d2570d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:54.820527 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc4e8d-3961-42e2-b8f0-c335d0d2570d service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:54.832118 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1085/2188] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:55:54 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:54.833296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac786a8f-33c5-4af8-b378-9c59fa3a6dfc admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:54.834407 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1104/2189] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:55:54.908300 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac786a8f-33c5-4af8-b378-9c59fa3a6dfc admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:54.915429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8744a-48f6-4304-99b8-bf2da9fede61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:54.916403 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8744a-48f6-4304-99b8-bf2da9fede61 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:54.916786 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8744a-48f6-4304-99b8-bf2da9fede61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:54.916987 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8744a-48f6-4304-99b8-bf2da9fede61 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:54.917658 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8744a-48f6-4304-99b8-bf2da9fede61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:54.920318 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8744a-48f6-4304-99b8-bf2da9fede61 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:54.946398 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1086/2190] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:55:54 2022] POST /identity/v3/users => generated 283 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:55:54.955877 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc4e8d-3961-42e2-b8f0-c335d0d2570d service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:54.977645 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc4e8d-3961-42e2-b8f0-c335d0d2570d service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:54.983881 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa1c17-42b3-45a5-8f05-c88d8d82514d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:54.984504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa1c17-42b3-45a5-8f05-c88d8d82514d admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:54.984827 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa1c17-42b3-45a5-8f05-c88d8d82514d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:54.985100 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa1c17-42b3-45a5-8f05-c88d8d82514d admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:54.985741 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa1c17-42b3-45a5-8f05-c88d8d82514d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:54.988205 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa1c17-42b3-45a5-8f05-c88d8d82514d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:55.000232 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1105/2191] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:55.004679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8744a-48f6-4304-99b8-bf2da9fede61 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:55.016153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8744a-48f6-4304-99b8-bf2da9fede61 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:55.021648 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f613e-4d4d-4733-8348-d455b88de170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:55.022261 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f613e-4d4d-4733-8348-d455b88de170 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:55.022544 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f613e-4d4d-4733-8348-d455b88de170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:55.022818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f613e-4d4d-4733-8348-d455b88de170 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:55.023459 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f613e-4d4d-4733-8348-d455b88de170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:55.025829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f613e-4d4d-4733-8348-d455b88de170 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:55.040296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1087/2192] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:55:55.043490 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa1c17-42b3-45a5-8f05-c88d8d82514d admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:55.062707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa1c17-42b3-45a5-8f05-c88d8d82514d admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:55.070275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335a3a9-349d-45b8-9ae6-26dc7bd67c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:55.071664 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335a3a9-349d-45b8-9ae6-26dc7bd67c14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:55.071664 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335a3a9-349d-45b8-9ae6-26dc7bd67c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:55.073799 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335a3a9-349d-45b8-9ae6-26dc7bd67c14 admin admin] PATH_INFO: `/v3/projects/1dad9fc8c15745098c5ff3e50cb024f4/users/512e942a8de048ba94324f4f6295aaf2/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:55.094792 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335a3a9-349d-45b8-9ae6-26dc7bd67c14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dad9fc8c15745098c5ff3e50cb024f4, user_id=512e942a8de048ba94324f4f6295aaf2, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:55.098392 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335a3a9-349d-45b8-9ae6-26dc7bd67c14 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:55.127037 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1106/2193] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:55 2022] PUT /identity/v3/projects/1dad9fc8c15745098c5ff3e50cb024f4/users/512e942a8de048ba94324f4f6295aaf2/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:55.139283 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdc0d7-18db-4a6e-a5c3-e1bfde127156 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:55.139524 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdc0d7-18db-4a6e-a5c3-e1bfde127156 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:55.139887 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdc0d7-18db-4a6e-a5c3-e1bfde127156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:55.193167 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-80fdc0d7-18db-4a6e-a5c3-e1bfde127156 None None] MFA Rules not processed for user `512e942a8de048ba94324f4f6295aaf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:55.194390 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-80fdc0d7-18db-4a6e-a5c3-e1bfde127156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:55.201846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335a3a9-349d-45b8-9ae6-26dc7bd67c14 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:55.203632 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-a335a3a9-349d-45b8-9ae6-26dc7bd67c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:55.209378 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:55.370477 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1088/2194] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:55.375946 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdc0d7-18db-4a6e-a5c3-e1bfde127156 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:55.376286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335a3a9-349d-45b8-9ae6-26dc7bd67c14 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:55.383302 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c8c50-9f5f-4ecb-9d88-07a02bd8e30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:55.384226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c8c50-9f5f-4ecb-9d88-07a02bd8e30d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:55.384805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c8c50-9f5f-4ecb-9d88-07a02bd8e30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:55.385760 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c8c50-9f5f-4ecb-9d88-07a02bd8e30d admin admin] PATH_INFO: `/v3/projects/4b0c6a35b8964637b86329ab2a75e95e` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:55.387263 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c8c50-9f5f-4ecb-9d88-07a02bd8e30d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b0c6a35b8964637b86329ab2a75e95e)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:55.389755 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c8c50-9f5f-4ecb-9d88-07a02bd8e30d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:55.391035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdc0d7-18db-4a6e-a5c3-e1bfde127156 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:55.398204 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93e2f2-05e3-429a-8ab7-55165fa342bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:55.398720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93e2f2-05e3-429a-8ab7-55165fa342bd admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:55.399068 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93e2f2-05e3-429a-8ab7-55165fa342bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:55.399269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93e2f2-05e3-429a-8ab7-55165fa342bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:55.399947 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93e2f2-05e3-429a-8ab7-55165fa342bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:55.401872 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93e2f2-05e3-429a-8ab7-55165fa342bd admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:55.448083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1089/2195] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:55:55 2022] POST /identity/v3/projects => generated 367 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:55.452944 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93e2f2-05e3-429a-8ab7-55165fa342bd admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:55.459033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:55.465126 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e28c8c50-9f5f-4ecb-9d88-07a02bd8e30d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b0c6a35b8964637b86329ab2a75e95e', 'request_id': 'req-e28c8c50-9f5f-4ecb-9d88-07a02bd8e30d'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:55.484458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e28c8c50-9f5f-4ecb-9d88-07a02bd8e30d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b0c6a35b8964637b86329ab2a75e95e', 'request_id': 'req-e28c8c50-9f5f-4ecb-9d88-07a02bd8e30d'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:55.496332 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e28c8c50-9f5f-4ecb-9d88-07a02bd8e30d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b0c6a35b8964637b86329ab2a75e95e', 'request_id': 'req-e28c8c50-9f5f-4ecb-9d88-07a02bd8e30d'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:55.520251 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e28c8c50-9f5f-4ecb-9d88-07a02bd8e30d admin admin] The token cache is being invalidate because project 4b0c6a35b8964637b86329ab2a75e95e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:55:55.520683 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e28c8c50-9f5f-4ecb-9d88-07a02bd8e30d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:55.521748 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e28c8c50-9f5f-4ecb-9d88-07a02bd8e30d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:55:55.526961 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1107/2196] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:55:55 2022] DELETE /identity/v3/projects/4b0c6a35b8964637b86329ab2a75e95e => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:55:55.529122 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790313c8-23e5-413b-bfd7-21987095960d None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:55.529803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790313c8-23e5-413b-bfd7-21987095960d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:55.530177 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790313c8-23e5-413b-bfd7-21987095960d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:55.575784 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-790313c8-23e5-413b-bfd7-21987095960d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:55.577164 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-790313c8-23e5-413b-bfd7-21987095960d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:55.601874 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93e2f2-05e3-429a-8ab7-55165fa342bd admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:55.606392 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb64e38-7696-4b36-acf3-28e527eb6009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:55.607318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1108/2197] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:55.607653 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb64e38-7696-4b36-acf3-28e527eb6009 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:55.608037 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb64e38-7696-4b36-acf3-28e527eb6009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:55.608314 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb64e38-7696-4b36-acf3-28e527eb6009 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:55.609103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb64e38-7696-4b36-acf3-28e527eb6009 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:55.611312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb64e38-7696-4b36-acf3-28e527eb6009 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:55.648628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1090/2198] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:55:55 2022] POST /identity/v3/users => generated 290 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:55.651938 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790313c8-23e5-413b-bfd7-21987095960d None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:55.653624 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-790313c8-23e5-413b-bfd7-21987095960d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:55.687005 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961101ed-0fba-44c3-a330-78bf1a62efee None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:55.687290 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961101ed-0fba-44c3-a330-78bf1a62efee None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:55.687583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961101ed-0fba-44c3-a330-78bf1a62efee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:55.694091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790313c8-23e5-413b-bfd7-21987095960d None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:55.704517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e388a0-4447-448d-94fb-6e69b94ac780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:55.707139 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e388a0-4447-448d-94fb-6e69b94ac780 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:55.707765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e388a0-4447-448d-94fb-6e69b94ac780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:55.708229 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e388a0-4447-448d-94fb-6e69b94ac780 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:55.709226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e388a0-4447-448d-94fb-6e69b94ac780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:55.711616 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e388a0-4447-448d-94fb-6e69b94ac780 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:55.725427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1109/2199] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:55.727595 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-961101ed-0fba-44c3-a330-78bf1a62efee None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:55.728937 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-961101ed-0fba-44c3-a330-78bf1a62efee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:55.730128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e388a0-4447-448d-94fb-6e69b94ac780 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:55.738135 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1091/2200] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:55:55.741866 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961101ed-0fba-44c3-a330-78bf1a62efee None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:55.752477 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961101ed-0fba-44c3-a330-78bf1a62efee None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:55.752683 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e388a0-4447-448d-94fb-6e69b94ac780 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:55.757897 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9743388-6afa-4622-a1b3-82db9070875e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:55.758903 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9743388-6afa-4622-a1b3-82db9070875e admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:55.759109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb4081-a799-4271-8dae-29c6f5863645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:55.759292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9743388-6afa-4622-a1b3-82db9070875e admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:55.759679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9743388-6afa-4622-a1b3-82db9070875e admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:55.759944 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb4081-a799-4271-8dae-29c6f5863645 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:55.760481 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb4081-a799-4271-8dae-29c6f5863645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:55.760770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb4081-a799-4271-8dae-29c6f5863645 admin admin] PATH_INFO: `/v3/projects/b779177592bc4cfd8a21dcd8305fd690/users/19f499db42ea489ebe4d5d184d6e200e/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:55.761093 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9743388-6afa-4622-a1b3-82db9070875e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:55:55.761462 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9743388-6afa-4622-a1b3-82db9070875e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:55.763332 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9743388-6afa-4622-a1b3-82db9070875e admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:55.794959 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1092/2201] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:55:55.801702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9743388-6afa-4622-a1b3-82db9070875e admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:55.809937 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb4081-a799-4271-8dae-29c6f5863645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b779177592bc4cfd8a21dcd8305fd690, user_id=19f499db42ea489ebe4d5d184d6e200e, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:55.812679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9743388-6afa-4622-a1b3-82db9070875e admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:55.813094 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb4081-a799-4271-8dae-29c6f5863645 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:55.817944 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774c0e6-4bf3-49c9-9a99-471a0f51c139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:55.818837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774c0e6-4bf3-49c9-9a99-471a0f51c139 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:55.819458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774c0e6-4bf3-49c9-9a99-471a0f51c139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:55.820068 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774c0e6-4bf3-49c9-9a99-471a0f51c139 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:55.821151 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774c0e6-4bf3-49c9-9a99-471a0f51c139 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:55.823535 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774c0e6-4bf3-49c9-9a99-471a0f51c139 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:55.833253 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1110/2202] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:55 2022] PUT /identity/v3/projects/b779177592bc4cfd8a21dcd8305fd690/users/19f499db42ea489ebe4d5d184d6e200e/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:55.837007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb4081-a799-4271-8dae-29c6f5863645 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:55.860378 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1093/2203] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:55:55 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:55.864626 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774c0e6-4bf3-49c9-9a99-471a0f51c139 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:55.914779 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:55.930125 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb4081-a799-4271-8dae-29c6f5863645 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:55.951183 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774c0e6-4bf3-49c9-9a99-471a0f51c139 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:55.958079 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbea19-ab9f-4c48-ad8b-74446cd54b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:55.959660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbea19-ab9f-4c48-ad8b-74446cd54b22 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:55.961417 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbea19-ab9f-4c48-ad8b-74446cd54b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:55.961957 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbea19-ab9f-4c48-ad8b-74446cd54b22 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:55.962834 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbea19-ab9f-4c48-ad8b-74446cd54b22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:55.965209 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbea19-ab9f-4c48-ad8b-74446cd54b22 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:56.001461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1094/2204] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:55:55 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:55:56.006182 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbea19-ab9f-4c48-ad8b-74446cd54b22 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:56.011768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549de282-eed5-42c9-b264-1e17945c03f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:56.012393 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549de282-eed5-42c9-b264-1e17945c03f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:56.012673 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549de282-eed5-42c9-b264-1e17945c03f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:56.012951 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549de282-eed5-42c9-b264-1e17945c03f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:56.013595 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549de282-eed5-42c9-b264-1e17945c03f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:56.015339 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549de282-eed5-42c9-b264-1e17945c03f9 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:56.021597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbea19-ab9f-4c48-ad8b-74446cd54b22 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:56.027110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ab4e2-7939-4f48-ad2d-be447d64e8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:56.027850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ab4e2-7939-4f48-ad2d-be447d64e8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:56.028083 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1111/2205] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:55:56.028517 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ab4e2-7939-4f48-ad2d-be447d64e8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:56.028913 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ab4e2-7939-4f48-ad2d-be447d64e8c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:56.029677 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ab4e2-7939-4f48-ad2d-be447d64e8c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:56.031542 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ab4e2-7939-4f48-ad2d-be447d64e8c2 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:56.032444 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549de282-eed5-42c9-b264-1e17945c03f9 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:56.044420 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549de282-eed5-42c9-b264-1e17945c03f9 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:56.045273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1095/2206] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:56.049224 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ab4e2-7939-4f48-ad2d-be447d64e8c2 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:56.050432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e8214-d8a1-4472-9fdb-df7e45d35184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:56.051320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e8214-d8a1-4472-9fdb-df7e45d35184 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:56.051811 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e8214-d8a1-4472-9fdb-df7e45d35184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:56.052261 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e8214-d8a1-4472-9fdb-df7e45d35184 admin admin] PATH_INFO: `/v3/projects/b779177592bc4cfd8a21dcd8305fd690/users/19f499db42ea489ebe4d5d184d6e200e/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:56.054617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e8214-d8a1-4472-9fdb-df7e45d35184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b779177592bc4cfd8a21dcd8305fd690, user_id=19f499db42ea489ebe4d5d184d6e200e, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:56.057779 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e8214-d8a1-4472-9fdb-df7e45d35184 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:56.061870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ab4e2-7939-4f48-ad2d-be447d64e8c2 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:56.069587 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5710b7ec-c13d-4052-af32-b3fef30e0f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:56.070201 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710b7ec-c13d-4052-af32-b3fef30e0f3a admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:56.070475 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710b7ec-c13d-4052-af32-b3fef30e0f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:56.071860 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710b7ec-c13d-4052-af32-b3fef30e0f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:56.074520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1112/2207] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:56 2022] PUT /identity/v3/projects/b779177592bc4cfd8a21dcd8305fd690/users/19f499db42ea489ebe4d5d184d6e200e/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:55:56.083520 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5710b7ec-c13d-4052-af32-b3fef30e0f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:56.083520 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5710b7ec-c13d-4052-af32-b3fef30e0f3a admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:56.084992 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d6da9-489c-4cdf-88db-827fbe558cdb None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:56.085676 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d6da9-489c-4cdf-88db-827fbe558cdb None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:56.085793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d6da9-489c-4cdf-88db-827fbe558cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:56.094304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1096/2208] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:55:56.106518 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5710b7ec-c13d-4052-af32-b3fef30e0f3a admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:56.121604 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5710b7ec-c13d-4052-af32-b3fef30e0f3a admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:56.129907 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-256d6da9-489c-4cdf-88db-827fbe558cdb None None] MFA Rules not processed for user `19f499db42ea489ebe4d5d184d6e200e`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:56.131601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-256d6da9-489c-4cdf-88db-827fbe558cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:56.141994 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:56.209014 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c5be6-aa68-4777-83e5-6478e56d6fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:56.209704 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c5be6-aa68-4777-83e5-6478e56d6fec admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:56.209980 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c5be6-aa68-4777-83e5-6478e56d6fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:56.210255 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c5be6-aa68-4777-83e5-6478e56d6fec admin admin] PATH_INFO: `/v3/projects/2ccac39e68324aaeb32d5eb06b8d685c/users/daf8638ae54f4c049a1f8879e081590b/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:56.231459 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c5be6-aa68-4777-83e5-6478e56d6fec admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ccac39e68324aaeb32d5eb06b8d685c, user_id=daf8638ae54f4c049a1f8879e081590b, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:56.234222 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c5be6-aa68-4777-83e5-6478e56d6fec admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:56.256018 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1097/2209] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:56 2022] PUT /identity/v3/projects/2ccac39e68324aaeb32d5eb06b8d685c/users/daf8638ae54f4c049a1f8879e081590b/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:56.256218 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1113/2210] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:55:56.267519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65d392-f8c4-479b-b535-873f851377b1 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:56.267885 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65d392-f8c4-479b-b535-873f851377b1 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:56.268188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65d392-f8c4-479b-b535-873f851377b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:56.270805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d6da9-489c-4cdf-88db-827fbe558cdb None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:56.271520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-256d6da9-489c-4cdf-88db-827fbe558cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:56.318941 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-9e65d392-f8c4-479b-b535-873f851377b1 None None] MFA Rules not processed for user `daf8638ae54f4c049a1f8879e081590b`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:56.320570 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9e65d392-f8c4-479b-b535-873f851377b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:56.371118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d6da9-489c-4cdf-88db-827fbe558cdb None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:56.418994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1098/2211] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:56.423452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65d392-f8c4-479b-b535-873f851377b1 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:56.441865 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:56.482193 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c132aee-13ca-4472-9fc2-e5c06d1daa13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:56.482944 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c132aee-13ca-4472-9fc2-e5c06d1daa13 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:56.483249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c132aee-13ca-4472-9fc2-e5c06d1daa13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:56.483603 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c132aee-13ca-4472-9fc2-e5c06d1daa13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:56.529339 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65d392-f8c4-479b-b535-873f851377b1 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:56.535442 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5beda3f-2285-4acf-a2ab-926ba740e75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:56.536251 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5beda3f-2285-4acf-a2ab-926ba740e75c admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:56.536625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5beda3f-2285-4acf-a2ab-926ba740e75c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:56.537317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5beda3f-2285-4acf-a2ab-926ba740e75c admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:56.538622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5beda3f-2285-4acf-a2ab-926ba740e75c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:56.542504 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5beda3f-2285-4acf-a2ab-926ba740e75c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:56.580020 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1099/2212] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:55:56 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:55:56.584000 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c132aee-13ca-4472-9fc2-e5c06d1daa13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:56.585669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c132aee-13ca-4472-9fc2-e5c06d1daa13 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:56.598693 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5beda3f-2285-4acf-a2ab-926ba740e75c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:56.599407 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1114/2213] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:55:56.697230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5beda3f-2285-4acf-a2ab-926ba740e75c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:56.704591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d53a1a-5f47-48f0-9e83-f2c9314f379c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:56.704591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d53a1a-5f47-48f0-9e83-f2c9314f379c admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:56.704591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d53a1a-5f47-48f0-9e83-f2c9314f379c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:56.704986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d53a1a-5f47-48f0-9e83-f2c9314f379c admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:56.705699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d53a1a-5f47-48f0-9e83-f2c9314f379c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:56.707661 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d53a1a-5f47-48f0-9e83-f2c9314f379c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:56.732672 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1100/2214] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:55:56 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:56.736094 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c132aee-13ca-4472-9fc2-e5c06d1daa13 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:56.742085 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d53a1a-5f47-48f0-9e83-f2c9314f379c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:56.744224 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-33d53a1a-5f47-48f0-9e83-f2c9314f379c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:56.764902 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c132aee-13ca-4472-9fc2-e5c06d1daa13 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:56.770115 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cb449-0deb-4681-82ea-430188d9f0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:56.770802 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cb449-0deb-4681-82ea-430188d9f0e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:56.771141 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cb449-0deb-4681-82ea-430188d9f0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:56.771475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cb449-0deb-4681-82ea-430188d9f0e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:56.772207 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106cb449-0deb-4681-82ea-430188d9f0e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:56.773958 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106cb449-0deb-4681-82ea-430188d9f0e5 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:56.779875 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:56.787097 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1115/2215] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:56.790893 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cb449-0deb-4681-82ea-430188d9f0e5 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:56.801979 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cb449-0deb-4681-82ea-430188d9f0e5 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:56.807457 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08882c-1580-481d-a8d7-0949d249d7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:56.808094 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08882c-1580-481d-a8d7-0949d249d7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:56.808378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08882c-1580-481d-a8d7-0949d249d7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:56.808641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08882c-1580-481d-a8d7-0949d249d7b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:56.809276 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab08882c-1580-481d-a8d7-0949d249d7b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:56.811011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab08882c-1580-481d-a8d7-0949d249d7b4 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:56.821950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1116/2216] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:56.822500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d53a1a-5f47-48f0-9e83-f2c9314f379c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:56.825869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08882c-1580-481d-a8d7-0949d249d7b4 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:56.828666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6931c10-d44f-4607-ac01-513a239783fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:56.829489 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6931c10-d44f-4607-ac01-513a239783fa service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:56.829774 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6931c10-d44f-4607-ac01-513a239783fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:56.830052 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6931c10-d44f-4607-ac01-513a239783fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:56.832773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-f6931c10-d44f-4607-ac01-513a239783fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:56.837088 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08882c-1580-481d-a8d7-0949d249d7b4 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:56.842485 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261e622-9741-4dc6-9a95-f39d7fad4c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:56.843151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261e622-9741-4dc6-9a95-f39d7fad4c48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:56.843740 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261e622-9741-4dc6-9a95-f39d7fad4c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:56.844024 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261e622-9741-4dc6-9a95-f39d7fad4c48 admin admin] PATH_INFO: `/v3/projects/22b1508f8a6d4dc5867d07508f766c30/users/2d34bd934b174453bc439af428948441/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:56.863029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261e622-9741-4dc6-9a95-f39d7fad4c48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22b1508f8a6d4dc5867d07508f766c30, user_id=2d34bd934b174453bc439af428948441, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:56.866206 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261e622-9741-4dc6-9a95-f39d7fad4c48 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:56.885709 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1117/2217] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:56 2022] PUT /identity/v3/projects/22b1508f8a6d4dc5867d07508f766c30/users/2d34bd934b174453bc439af428948441/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:55:56.896244 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:56.897530 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8f233-7a8c-4ab1-967b-a0da72d568a2 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:56.898227 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8f233-7a8c-4ab1-967b-a0da72d568a2 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:56.898582 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8f233-7a8c-4ab1-967b-a0da72d568a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:56.918932 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6931c10-d44f-4607-ac01-513a239783fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:56.920516 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6931c10-d44f-4607-ac01-513a239783fa service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:56.934440 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1101/2218] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:55:56.941613 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-56c8f233-7a8c-4ab1-967b-a0da72d568a2 None None] MFA Rules not processed for user `2d34bd934b174453bc439af428948441`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:56.942839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-56c8f233-7a8c-4ab1-967b-a0da72d568a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:57.032170 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1118/2219] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:57.037497 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6931c10-d44f-4607-ac01-513a239783fa service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:57.053440 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6931c10-d44f-4607-ac01-513a239783fa service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:57.143093 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b004126-62b9-43bd-9bd4-7698df9e474a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:57.144677 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b004126-62b9-43bd-9bd4-7698df9e474a admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:57.144677 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b004126-62b9-43bd-9bd4-7698df9e474a admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:57.145032 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b004126-62b9-43bd-9bd4-7698df9e474a admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:57.146362 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b004126-62b9-43bd-9bd4-7698df9e474a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:57.150742 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b004126-62b9-43bd-9bd4-7698df9e474a admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:57.213529 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1102/2220] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:55:57 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:55:57.214012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8f233-7a8c-4ab1-967b-a0da72d568a2 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:57.234607 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8f233-7a8c-4ab1-967b-a0da72d568a2 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:57.365442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb17eda-c635-45b1-a9dc-ebbe3caa2e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:57.366283 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb17eda-c635-45b1-a9dc-ebbe3caa2e51 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:57.366743 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb17eda-c635-45b1-a9dc-ebbe3caa2e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:57.367188 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb17eda-c635-45b1-a9dc-ebbe3caa2e51 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:57.368049 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb17eda-c635-45b1-a9dc-ebbe3caa2e51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:57.370253 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb17eda-c635-45b1-a9dc-ebbe3caa2e51 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:57.437611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1119/2221] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:55:57 2022] POST /identity/v3/users => generated 298 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:57.440730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b004126-62b9-43bd-9bd4-7698df9e474a admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:57.454872 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b004126-62b9-43bd-9bd4-7698df9e474a admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:57.472332 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9fed3-fa0d-40ec-aa3a-ed60048d2fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:57.473136 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9fed3-fa0d-40ec-aa3a-ed60048d2fef admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:57.473485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9fed3-fa0d-40ec-aa3a-ed60048d2fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:57.473837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9fed3-fa0d-40ec-aa3a-ed60048d2fef admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:57.474700 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda9fed3-fa0d-40ec-aa3a-ed60048d2fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:57.476764 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda9fed3-fa0d-40ec-aa3a-ed60048d2fef admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:57.503829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1103/2222] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:57.505102 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb17eda-c635-45b1-a9dc-ebbe3caa2e51 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:57.516367 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb17eda-c635-45b1-a9dc-ebbe3caa2e51 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:57.525520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144f884-4308-4a0b-b1a9-e12e47391959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:57.526296 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144f884-4308-4a0b-b1a9-e12e47391959 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:57.526650 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144f884-4308-4a0b-b1a9-e12e47391959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:57.526995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144f884-4308-4a0b-b1a9-e12e47391959 admin admin] PATH_INFO: `/v3/projects/0112254eb831424bbce289c60151b9ae/users/16a2a7ec18e74aedbbd94d6d745fef07/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:57.548011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144f884-4308-4a0b-b1a9-e12e47391959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0112254eb831424bbce289c60151b9ae, user_id=16a2a7ec18e74aedbbd94d6d745fef07, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:57.551890 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144f884-4308-4a0b-b1a9-e12e47391959 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:57.574557 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9fed3-fa0d-40ec-aa3a-ed60048d2fef admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:57.575653 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1120/2223] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:57 2022] PUT /identity/v3/projects/0112254eb831424bbce289c60151b9ae/users/16a2a7ec18e74aedbbd94d6d745fef07/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:55:57.592013 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9fed3-fa0d-40ec-aa3a-ed60048d2fef admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:57.673031 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c36ed-1e4e-4a0b-9585-fbcd663bcdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:57.674787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c36ed-1e4e-4a0b-9585-fbcd663bcdbb admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:57.675275 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c36ed-1e4e-4a0b-9585-fbcd663bcdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:57.675775 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c36ed-1e4e-4a0b-9585-fbcd663bcdbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:57.676632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c36ed-1e4e-4a0b-9585-fbcd663bcdbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:57.678691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c36ed-1e4e-4a0b-9585-fbcd663bcdbb admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:57.690720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1104/2224] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:57.695851 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144f884-4308-4a0b-b1a9-e12e47391959 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:57.707109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144f884-4308-4a0b-b1a9-e12e47391959 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:57.711972 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cb408-6a37-49a9-b848-dd447341c107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:57.712648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cb408-6a37-49a9-b848-dd447341c107 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:57.713073 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cb408-6a37-49a9-b848-dd447341c107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:57.713353 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cb408-6a37-49a9-b848-dd447341c107 admin admin] PATH_INFO: `/v3/projects/0112254eb831424bbce289c60151b9ae/users/16a2a7ec18e74aedbbd94d6d745fef07/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:57.715271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cb408-6a37-49a9-b848-dd447341c107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0112254eb831424bbce289c60151b9ae, user_id=16a2a7ec18e74aedbbd94d6d745fef07, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:57.718661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cb408-6a37-49a9-b848-dd447341c107 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:57.735845 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1121/2225] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:57 2022] PUT /identity/v3/projects/0112254eb831424bbce289c60151b9ae/users/16a2a7ec18e74aedbbd94d6d745fef07/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:57.748401 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e603b-9a47-4354-a040-98786a92a964 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:57.748927 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e603b-9a47-4354-a040-98786a92a964 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:57.748927 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e603b-9a47-4354-a040-98786a92a964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:57.780314 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-502e603b-9a47-4354-a040-98786a92a964 None None] MFA Rules not processed for user `16a2a7ec18e74aedbbd94d6d745fef07`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:57.781835 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-502e603b-9a47-4354-a040-98786a92a964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:57.826738 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:57.860650 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1105/2226] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:57.873162 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cb408-6a37-49a9-b848-dd447341c107 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:57.883149 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:57.982583 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cb408-6a37-49a9-b848-dd447341c107 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:57.995557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c9ea3-fb21-4367-99e6-7a64ddacdd54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:57.996224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c9ea3-fb21-4367-99e6-7a64ddacdd54 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:57.996508 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c9ea3-fb21-4367-99e6-7a64ddacdd54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:57.996779 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c9ea3-fb21-4367-99e6-7a64ddacdd54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:58.076861 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7c9ea3-fb21-4367-99e6-7a64ddacdd54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:58.078723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7c9ea3-fb21-4367-99e6-7a64ddacdd54 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:58.098105 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1122/2227] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:55:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:55:58.183145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e05d2-bffd-4f55-8615-062adccd6433 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:58.183433 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e05d2-bffd-4f55-8615-062adccd6433 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:58.183737 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e05d2-bffd-4f55-8615-062adccd6433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:58.214847 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-724e05d2-bffd-4f55-8615-062adccd6433 None None] MFA Rules not processed for user `933f34b82d5b4355844a57707de417a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:55:58.216258 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-724e05d2-bffd-4f55-8615-062adccd6433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:58.295094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1106/2228] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:55:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:55:58.296838 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c9ea3-fb21-4367-99e6-7a64ddacdd54 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:58.298726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ed7c9ea3-fb21-4367-99e6-7a64ddacdd54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:55:58.392014 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c9ea3-fb21-4367-99e6-7a64ddacdd54 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:58.397966 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab42a78-fc48-4615-9cd9-f68d59dbae05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:58.398602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab42a78-fc48-4615-9cd9-f68d59dbae05 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:58.398880 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab42a78-fc48-4615-9cd9-f68d59dbae05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:58.399149 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab42a78-fc48-4615-9cd9-f68d59dbae05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:58.399828 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab42a78-fc48-4615-9cd9-f68d59dbae05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:58.401750 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab42a78-fc48-4615-9cd9-f68d59dbae05 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:58.444953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1123/2229] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:55:58 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:55:58.449070 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e05d2-bffd-4f55-8615-062adccd6433 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:58.463652 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e05d2-bffd-4f55-8615-062adccd6433 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:58.572243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219e093-9672-4dc8-9326-b3d9f0a5bee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:58.572864 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219e093-9672-4dc8-9326-b3d9f0a5bee9 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:58.573146 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219e093-9672-4dc8-9326-b3d9f0a5bee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:58.573418 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219e093-9672-4dc8-9326-b3d9f0a5bee9 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:58.574054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219e093-9672-4dc8-9326-b3d9f0a5bee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:58.576005 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219e093-9672-4dc8-9326-b3d9f0a5bee9 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:58.598830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1107/2230] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:55:58 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:55:58.602801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab42a78-fc48-4615-9cd9-f68d59dbae05 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:58.613172 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab42a78-fc48-4615-9cd9-f68d59dbae05 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:58.617830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdfded-9f84-4eab-addb-1d5766b82592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:58.618436 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdfded-9f84-4eab-addb-1d5766b82592 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:58.618717 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdfded-9f84-4eab-addb-1d5766b82592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:58.618984 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdfded-9f84-4eab-addb-1d5766b82592 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:58.619645 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bdfded-9f84-4eab-addb-1d5766b82592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:58.621274 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bdfded-9f84-4eab-addb-1d5766b82592 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:58.632889 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1124/2231] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:55:58.637354 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219e093-9672-4dc8-9326-b3d9f0a5bee9 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:58.648945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219e093-9672-4dc8-9326-b3d9f0a5bee9 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:58.654732 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c91f4-452d-4784-b4e7-f3b400ceae74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:58.655511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c91f4-452d-4784-b4e7-f3b400ceae74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:58.655825 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c91f4-452d-4784-b4e7-f3b400ceae74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:58.656099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c91f4-452d-4784-b4e7-f3b400ceae74 admin admin] PATH_INFO: `/v3/projects/ab91077f61f24a8f85c33e2ea37c6de7/users/d8bf07ba081e40e98f17d94749b1cae0/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:58.673102 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c91f4-452d-4784-b4e7-f3b400ceae74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab91077f61f24a8f85c33e2ea37c6de7, user_id=d8bf07ba081e40e98f17d94749b1cae0, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:58.676395 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c91f4-452d-4784-b4e7-f3b400ceae74 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:58.690740 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1108/2232] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:58 2022] PUT /identity/v3/projects/ab91077f61f24a8f85c33e2ea37c6de7/users/d8bf07ba081e40e98f17d94749b1cae0/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:55:58.694919 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdfded-9f84-4eab-addb-1d5766b82592 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:58.705044 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdfded-9f84-4eab-addb-1d5766b82592 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:58.737139 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:55:58.761840 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4921be-9d6c-4c27-ad52-89c5e1e3a19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:58.762443 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4921be-9d6c-4c27-ad52-89c5e1e3a19a admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:58.762719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4921be-9d6c-4c27-ad52-89c5e1e3a19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:58.762992 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4921be-9d6c-4c27-ad52-89c5e1e3a19a admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:58.763649 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4921be-9d6c-4c27-ad52-89c5e1e3a19a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:58.765247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4921be-9d6c-4c27-ad52-89c5e1e3a19a admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:58.775077 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1125/2233] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:55:58.779145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c91f4-452d-4784-b4e7-f3b400ceae74 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:55:58.789836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c91f4-452d-4784-b4e7-f3b400ceae74 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:55:58.794429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf4a0d-eff8-4d26-8e2b-ac9600ad5bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:55:58.795069 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf4a0d-eff8-4d26-8e2b-ac9600ad5bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:55:58.795346 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf4a0d-eff8-4d26-8e2b-ac9600ad5bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:55:58.795641 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf4a0d-eff8-4d26-8e2b-ac9600ad5bc5 admin admin] PATH_INFO: `/v3/projects/ab91077f61f24a8f85c33e2ea37c6de7/users/d8bf07ba081e40e98f17d94749b1cae0/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:55:58.797558 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf4a0d-eff8-4d26-8e2b-ac9600ad5bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab91077f61f24a8f85c33e2ea37c6de7, user_id=d8bf07ba081e40e98f17d94749b1cae0, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:55:58.800771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf4a0d-eff8-4d26-8e2b-ac9600ad5bc5 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:55:58.810685 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1109/2234] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:55:58 2022] PUT /identity/v3/projects/ab91077f61f24a8f85c33e2ea37c6de7/users/d8bf07ba081e40e98f17d94749b1cae0/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:56:03.075035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02fe2c-2e7a-40f8-a3f2-2a2e4c2a8af0 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:03.075490 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02fe2c-2e7a-40f8-a3f2-2a2e4c2a8af0 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:03.075490 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02fe2c-2e7a-40f8-a3f2-2a2e4c2a8af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:03.114363 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-af02fe2c-2e7a-40f8-a3f2-2a2e4c2a8af0 None None] MFA Rules not processed for user `d8bf07ba081e40e98f17d94749b1cae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:56:03.115651 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-af02fe2c-2e7a-40f8-a3f2-2a2e4c2a8af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:03.239622 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1126/2235] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:56:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:56:03.252122 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf4a0d-eff8-4d26-8e2b-ac9600ad5bc5 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:03.254377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-4bdf4a0d-eff8-4d26-8e2b-ac9600ad5bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:03.289971 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:03.381971 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf4a0d-eff8-4d26-8e2b-ac9600ad5bc5 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:03.386377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33014a0d-cc72-400f-9972-ffd5c54e3e4d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce5ec441b06408bbadd6fc040af0da5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:03.387236 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33014a0d-cc72-400f-9972-ffd5c54e3e4d service glance] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:03.387647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33014a0d-cc72-400f-9972-ffd5c54e3e4d service glance] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:03.388015 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33014a0d-cc72-400f-9972-ffd5c54e3e4d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:03.492104 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33014a0d-cc72-400f-9972-ffd5c54e3e4d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:03.493557 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33014a0d-cc72-400f-9972-ffd5c54e3e4d service glance] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:03.504705 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1110/2236] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:56:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:56:03.660616 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21bae3-a187-43c7-b851-2d86387083ce None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:03.661068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21bae3-a187-43c7-b851-2d86387083ce None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:03.661405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21bae3-a187-43c7-b851-2d86387083ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:03.740586 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-4f21bae3-a187-43c7-b851-2d86387083ce None None] MFA Rules not processed for user `2346003afe74477496045e9bc72173d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:56:03.741877 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-4f21bae3-a187-43c7-b851-2d86387083ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:03.843233 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1127/2237] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Oct 21 21:56:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:56:03.896752 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33014a0d-cc72-400f-9972-ffd5c54e3e4d service glance] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:03.899283 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-33014a0d-cc72-400f-9972-ffd5c54e3e4d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:04.017363 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33014a0d-cc72-400f-9972-ffd5c54e3e4d service glance] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:04.022486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df96a64-ffb1-4c8a-b514-42cb378c4c1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ce720c8df014f9a8af062138aecd1b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:04.023101 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df96a64-ffb1-4c8a-b514-42cb378c4c1c service swift] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:04.023381 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df96a64-ffb1-4c8a-b514-42cb378c4c1c service swift] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:04.023681 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df96a64-ffb1-4c8a-b514-42cb378c4c1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:04.032092 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df96a64-ffb1-4c8a-b514-42cb378c4c1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:04.033710 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df96a64-ffb1-4c8a-b514-42cb378c4c1c service swift] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:04.042031 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1111/2238] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Oct 21 21:56:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 21:56:05.386663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21bae3-a187-43c7-b851-2d86387083ce None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:05.467339 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:05.543201 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21bae3-a187-43c7-b851-2d86387083ce None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:05.548375 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81953d88-3056-4b5d-bf48-24d857006cbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:05.549078 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81953d88-3056-4b5d-bf48-24d857006cbb service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:05.549354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81953d88-3056-4b5d-bf48-24d857006cbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:05.549625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81953d88-3056-4b5d-bf48-24d857006cbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:05.675626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81953d88-3056-4b5d-bf48-24d857006cbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:05.677725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81953d88-3056-4b5d-bf48-24d857006cbb service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:05.692678 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1128/2239] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:56:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:56:06.571950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df96a64-ffb1-4c8a-b514-42cb378c4c1c service swift] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:06.583130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df96a64-ffb1-4c8a-b514-42cb378c4c1c service swift] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:06.602709 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbec70-7c7f-4965-961c-f80c8b59ec67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:06.605010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbec70-7c7f-4965-961c-f80c8b59ec67 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:06.605367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbec70-7c7f-4965-961c-f80c8b59ec67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:06.605758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbec70-7c7f-4965-961c-f80c8b59ec67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:06.725696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:06.798754 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cbec70-7c7f-4965-961c-f80c8b59ec67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:06.800673 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cbec70-7c7f-4965-961c-f80c8b59ec67 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:06.815350 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1112/2240] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:56:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:56:09.646891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81953d88-3056-4b5d-bf48-24d857006cbb service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:09.788009 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81953d88-3056-4b5d-bf48-24d857006cbb service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:09.793135 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948836f-ea40-4f4c-a24d-1b04f8ef5f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:09.793770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948836f-ea40-4f4c-a24d-1b04f8ef5f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:09.794043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948836f-ea40-4f4c-a24d-1b04f8ef5f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:09.794309 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948836f-ea40-4f4c-a24d-1b04f8ef5f39 admin admin] PATH_INFO: `/v3/users/512e942a8de048ba94324f4f6295aaf2` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:09.796157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948836f-ea40-4f4c-a24d-1b04f8ef5f39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=512e942a8de048ba94324f4f6295aaf2)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:09.798967 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948836f-ea40-4f4c-a24d-1b04f8ef5f39 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:09.905757 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9948836f-ea40-4f4c-a24d-1b04f8ef5f39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '512e942a8de048ba94324f4f6295aaf2'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:09.927440 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9948836f-ea40-4f4c-a24d-1b04f8ef5f39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '512e942a8de048ba94324f4f6295aaf2'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:09.945732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9948836f-ea40-4f4c-a24d-1b04f8ef5f39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '512e942a8de048ba94324f4f6295aaf2'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:09.947201 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9948836f-ea40-4f4c-a24d-1b04f8ef5f39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '512e942a8de048ba94324f4f6295aaf2'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:09.956312 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:09.961874 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9948836f-ea40-4f4c-a24d-1b04f8ef5f39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '512e942a8de048ba94324f4f6295aaf2'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:09.973970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1129/2241] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:56:09 2022] DELETE /identity/v3/users/512e942a8de048ba94324f4f6295aaf2 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:56:09.978152 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a832d73-4ac8-4144-beba-0bd89dce2685 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:09.978692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a832d73-4ac8-4144-beba-0bd89dce2685 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:09.978692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a832d73-4ac8-4144-beba-0bd89dce2685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:10.023881 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-2a832d73-4ac8-4144-beba-0bd89dce2685 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:56:10.026532 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2a832d73-4ac8-4144-beba-0bd89dce2685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:10.138986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1113/2242] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:56:10.149898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948836f-ea40-4f4c-a24d-1b04f8ef5f39 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:10.152046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9948836f-ea40-4f4c-a24d-1b04f8ef5f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:10.269070 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948836f-ea40-4f4c-a24d-1b04f8ef5f39 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:10.274216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d46b94-6b70-482c-a9b2-23868370bef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:10.275020 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d46b94-6b70-482c-a9b2-23868370bef5 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:10.275498 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d46b94-6b70-482c-a9b2-23868370bef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:10.275846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d46b94-6b70-482c-a9b2-23868370bef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:10.289888 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d46b94-6b70-482c-a9b2-23868370bef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:10.291403 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d46b94-6b70-482c-a9b2-23868370bef5 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:10.312623 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1130/2243] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:56:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:56:10.624860 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a832d73-4ac8-4144-beba-0bd89dce2685 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:10.627080 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2a832d73-4ac8-4144-beba-0bd89dce2685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:10.650431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a832d73-4ac8-4144-beba-0bd89dce2685 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:10.665029 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785be2d-0add-4ba6-a350-b27edc6779e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:10.667987 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785be2d-0add-4ba6-a350-b27edc6779e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:10.668353 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785be2d-0add-4ba6-a350-b27edc6779e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:10.668717 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785be2d-0add-4ba6-a350-b27edc6779e1 admin admin] PATH_INFO: `/v3/projects/1dad9fc8c15745098c5ff3e50cb024f4` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:10.680514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785be2d-0add-4ba6-a350-b27edc6779e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dad9fc8c15745098c5ff3e50cb024f4)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:10.684343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785be2d-0add-4ba6-a350-b27edc6779e1 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:10.746157 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-d785be2d-0add-4ba6-a350-b27edc6779e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dad9fc8c15745098c5ff3e50cb024f4', 'request_id': 'req-d785be2d-0add-4ba6-a350-b27edc6779e1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:10.754278 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-d785be2d-0add-4ba6-a350-b27edc6779e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dad9fc8c15745098c5ff3e50cb024f4', 'request_id': 'req-d785be2d-0add-4ba6-a350-b27edc6779e1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:10.763243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-d785be2d-0add-4ba6-a350-b27edc6779e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dad9fc8c15745098c5ff3e50cb024f4', 'request_id': 'req-d785be2d-0add-4ba6-a350-b27edc6779e1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:10.777167 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-d785be2d-0add-4ba6-a350-b27edc6779e1 admin admin] The token cache is being invalidate because project 1dad9fc8c15745098c5ff3e50cb024f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:56:10.777466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-d785be2d-0add-4ba6-a350-b27edc6779e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:10.778323 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-d785be2d-0add-4ba6-a350-b27edc6779e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:10.780201 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1114/2244] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:56:10 2022] DELETE /identity/v3/projects/1dad9fc8c15745098c5ff3e50cb024f4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:56:10.784279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d46b94-6b70-482c-a9b2-23868370bef5 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:10.797311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d46b94-6b70-482c-a9b2-23868370bef5 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:10.870554 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136728d2-e737-4dbb-9b13-817d3fa73ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:10.871293 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136728d2-e737-4dbb-9b13-817d3fa73ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:10.871664 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136728d2-e737-4dbb-9b13-817d3fa73ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:10.872010 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136728d2-e737-4dbb-9b13-817d3fa73ead admin admin] PATH_INFO: `/v3/users/19f499db42ea489ebe4d5d184d6e200e` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:10.873429 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136728d2-e737-4dbb-9b13-817d3fa73ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=19f499db42ea489ebe4d5d184d6e200e)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:10.875448 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136728d2-e737-4dbb-9b13-817d3fa73ead admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:10.934831 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-136728d2-e737-4dbb-9b13-817d3fa73ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19f499db42ea489ebe4d5d184d6e200e'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:10.951752 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-136728d2-e737-4dbb-9b13-817d3fa73ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19f499db42ea489ebe4d5d184d6e200e'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:10.962247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-136728d2-e737-4dbb-9b13-817d3fa73ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19f499db42ea489ebe4d5d184d6e200e'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:10.963032 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-136728d2-e737-4dbb-9b13-817d3fa73ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19f499db42ea489ebe4d5d184d6e200e'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:10.972582 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-136728d2-e737-4dbb-9b13-817d3fa73ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19f499db42ea489ebe4d5d184d6e200e'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:10.979720 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1131/2245] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:56:10 2022] DELETE /identity/v3/users/19f499db42ea489ebe4d5d184d6e200e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:56:11.284946 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785be2d-0add-4ba6-a350-b27edc6779e1 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:11.293225 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d785be2d-0add-4ba6-a350-b27edc6779e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:11.307517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136728d2-e737-4dbb-9b13-817d3fa73ead admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:11.309221 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-136728d2-e737-4dbb-9b13-817d3fa73ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:11.314751 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785be2d-0add-4ba6-a350-b27edc6779e1 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:11.355989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:11.385720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:11.439260 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136728d2-e737-4dbb-9b13-817d3fa73ead admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:11.445590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea098bbf-ccc7-4908-bb47-70d88a360f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:11.446238 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea098bbf-ccc7-4908-bb47-70d88a360f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:11.447331 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea098bbf-ccc7-4908-bb47-70d88a360f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:11.447660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea098bbf-ccc7-4908-bb47-70d88a360f55 admin admin] PATH_INFO: `/v3/projects/b779177592bc4cfd8a21dcd8305fd690` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:11.448988 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea098bbf-ccc7-4908-bb47-70d88a360f55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b779177592bc4cfd8a21dcd8305fd690)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:11.452403 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea098bbf-ccc7-4908-bb47-70d88a360f55 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:11.453446 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaba39c-3648-45c6-9dc1-02192dc543c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:11.455323 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaba39c-3648-45c6-9dc1-02192dc543c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:11.455924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaba39c-3648-45c6-9dc1-02192dc543c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:11.456202 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaba39c-3648-45c6-9dc1-02192dc543c5 admin admin] PATH_INFO: `/v3/users/daf8638ae54f4c049a1f8879e081590b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:11.460899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaba39c-3648-45c6-9dc1-02192dc543c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daf8638ae54f4c049a1f8879e081590b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:11.465741 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaba39c-3648-45c6-9dc1-02192dc543c5 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:11.521282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ea098bbf-ccc7-4908-bb47-70d88a360f55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b779177592bc4cfd8a21dcd8305fd690', 'request_id': 'req-ea098bbf-ccc7-4908-bb47-70d88a360f55'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:11.538835 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ea098bbf-ccc7-4908-bb47-70d88a360f55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b779177592bc4cfd8a21dcd8305fd690', 'request_id': 'req-ea098bbf-ccc7-4908-bb47-70d88a360f55'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:11.549556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ea098bbf-ccc7-4908-bb47-70d88a360f55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b779177592bc4cfd8a21dcd8305fd690', 'request_id': 'req-ea098bbf-ccc7-4908-bb47-70d88a360f55'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:11.562869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ea098bbf-ccc7-4908-bb47-70d88a360f55 admin admin] The token cache is being invalidate because project b779177592bc4cfd8a21dcd8305fd690 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:56:11.564269 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ea098bbf-ccc7-4908-bb47-70d88a360f55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:11.565000 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ea098bbf-ccc7-4908-bb47-70d88a360f55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:11.567472 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1132/2246] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:56:11 2022] DELETE /identity/v3/projects/b779177592bc4cfd8a21dcd8305fd690 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:56:11.572174 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430eaaa5-4fd6-47c1-a177-47421219426c None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:11.572561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430eaaa5-4fd6-47c1-a177-47421219426c None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:11.572938 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430eaaa5-4fd6-47c1-a177-47421219426c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:11.590073 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-dbaba39c-3648-45c6-9dc1-02192dc543c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daf8638ae54f4c049a1f8879e081590b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:11.592146 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-dbaba39c-3648-45c6-9dc1-02192dc543c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daf8638ae54f4c049a1f8879e081590b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:11.599082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-430eaaa5-4fd6-47c1-a177-47421219426c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:56:11.603611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-430eaaa5-4fd6-47c1-a177-47421219426c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:11.613805 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-dbaba39c-3648-45c6-9dc1-02192dc543c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daf8638ae54f4c049a1f8879e081590b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:11.626366 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-dbaba39c-3648-45c6-9dc1-02192dc543c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daf8638ae54f4c049a1f8879e081590b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:11.627287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-dbaba39c-3648-45c6-9dc1-02192dc543c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daf8638ae54f4c049a1f8879e081590b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:11.648867 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1115/2247] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:56:11 2022] DELETE /identity/v3/users/daf8638ae54f4c049a1f8879e081590b => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:56:11.654287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb3aaf-e00b-4100-889b-b744bfb3d5d0 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:11.657047 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb3aaf-e00b-4100-889b-b744bfb3d5d0 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:11.657580 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb3aaf-e00b-4100-889b-b744bfb3d5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:11.710509 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-1dcb3aaf-e00b-4100-889b-b744bfb3d5d0 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:56:11.712418 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1dcb3aaf-e00b-4100-889b-b744bfb3d5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:11.791927 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1133/2248] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:56:11.791927 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bd384-5253-41d3-9689-3dd8d20c953f None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:11.791927 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bd384-5253-41d3-9689-3dd8d20c953f None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:11.791927 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bd384-5253-41d3-9689-3dd8d20c953f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:11.826803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-b81bd384-5253-41d3-9689-3dd8d20c953f None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:56:11.828188 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b81bd384-5253-41d3-9689-3dd8d20c953f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:11.835167 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1116/2249] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:56:11.838743 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1134/2250] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:56:11.842380 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb3aaf-e00b-4100-889b-b744bfb3d5d0 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:11.842614 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81bd384-5253-41d3-9689-3dd8d20c953f None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:11.843829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1dcb3aaf-e00b-4100-889b-b744bfb3d5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:11.855359 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81bd384-5253-41d3-9689-3dd8d20c953f None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:11.862415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c43e65-0e58-4826-974f-e7486781a828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:11.863141 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c43e65-0e58-4826-974f-e7486781a828 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:11.863278 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c43e65-0e58-4826-974f-e7486781a828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:11.863560 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c43e65-0e58-4826-974f-e7486781a828 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:11.864235 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c43e65-0e58-4826-974f-e7486781a828 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:56:11.864540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c43e65-0e58-4826-974f-e7486781a828 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:11.866193 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c43e65-0e58-4826-974f-e7486781a828 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:11.885842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1135/2251] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:56:11.886803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c43e65-0e58-4826-974f-e7486781a828 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:11.899072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c43e65-0e58-4826-974f-e7486781a828 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:11.910147 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa26702-8b71-4462-80e3-e626478ee3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:11.911360 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa26702-8b71-4462-80e3-e626478ee3bd admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:11.911897 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa26702-8b71-4462-80e3-e626478ee3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:11.912391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa26702-8b71-4462-80e3-e626478ee3bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:11.913237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa26702-8b71-4462-80e3-e626478ee3bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:11.915229 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa26702-8b71-4462-80e3-e626478ee3bd admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:11.927490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb3aaf-e00b-4100-889b-b744bfb3d5d0 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:11.931855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c7d50-8fdc-4281-8821-e7a5b445b8d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:11.932577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c7d50-8fdc-4281-8821-e7a5b445b8d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:11.933372 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c7d50-8fdc-4281-8821-e7a5b445b8d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:11.933769 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c7d50-8fdc-4281-8821-e7a5b445b8d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:11.947897 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c7d50-8fdc-4281-8821-e7a5b445b8d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:11.949745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c7d50-8fdc-4281-8821-e7a5b445b8d6 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:11.957380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1136/2252] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:56:11 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:56:11.959791 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa26702-8b71-4462-80e3-e626478ee3bd admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:11.964106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1117/2253] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:56:11.968135 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:12.039655 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa26702-8b71-4462-80e3-e626478ee3bd admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:12.047758 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94052b84-95dc-47ea-920f-756b1bf7d8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:12.049504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94052b84-95dc-47ea-920f-756b1bf7d8ac admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:12.049985 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94052b84-95dc-47ea-920f-756b1bf7d8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:12.050530 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94052b84-95dc-47ea-920f-756b1bf7d8ac admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:12.051403 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94052b84-95dc-47ea-920f-756b1bf7d8ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:12.054186 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94052b84-95dc-47ea-920f-756b1bf7d8ac admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:12.086377 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1137/2254] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:56:11 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:56:12.087689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c7d50-8fdc-4281-8821-e7a5b445b8d6 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:12.092786 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:12.100611 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c7d50-8fdc-4281-8821-e7a5b445b8d6 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:12.108443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b6676-81e5-4d6e-b617-dbbde70e45ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:12.109114 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b6676-81e5-4d6e-b617-dbbde70e45ff admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:12.109400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b6676-81e5-4d6e-b617-dbbde70e45ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:12.109690 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b6676-81e5-4d6e-b617-dbbde70e45ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:12.110318 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b6676-81e5-4d6e-b617-dbbde70e45ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:12.112865 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b6676-81e5-4d6e-b617-dbbde70e45ff admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:12.125642 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1118/2255] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:56:12.129721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94052b84-95dc-47ea-920f-756b1bf7d8ac admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:12.153696 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94052b84-95dc-47ea-920f-756b1bf7d8ac admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:12.163394 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4d496-d5c1-4514-b98c-2957169c6e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:12.163932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4d496-d5c1-4514-b98c-2957169c6e3e admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:12.164342 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4d496-d5c1-4514-b98c-2957169c6e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:12.164677 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4d496-d5c1-4514-b98c-2957169c6e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:12.165484 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4d496-d5c1-4514-b98c-2957169c6e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:12.167400 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4d496-d5c1-4514-b98c-2957169c6e3e admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:12.181164 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1138/2256] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:56:12.184817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b6676-81e5-4d6e-b617-dbbde70e45ff admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:12.199388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b6676-81e5-4d6e-b617-dbbde70e45ff admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:12.212716 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986e11c-3a07-41c5-aa7c-d06edd3c28a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:12.213569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986e11c-3a07-41c5-aa7c-d06edd3c28a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:12.214033 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986e11c-3a07-41c5-aa7c-d06edd3c28a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:12.214321 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986e11c-3a07-41c5-aa7c-d06edd3c28a5 admin admin] PATH_INFO: `/v3/projects/1c5f0bd0da4b4235b22eb4f4e0453034/users/b4116bba47f74b399a01831a66e1a213/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:12.237390 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986e11c-3a07-41c5-aa7c-d06edd3c28a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c5f0bd0da4b4235b22eb4f4e0453034, user_id=b4116bba47f74b399a01831a66e1a213, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:12.240386 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986e11c-3a07-41c5-aa7c-d06edd3c28a5 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:12.267874 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1119/2257] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:56:12 2022] PUT /identity/v3/projects/1c5f0bd0da4b4235b22eb4f4e0453034/users/b4116bba47f74b399a01831a66e1a213/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:56:12.271329 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12236489-8bf6-4c57-bb80-49af49f1c645 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:12.271725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12236489-8bf6-4c57-bb80-49af49f1c645 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:12.272218 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12236489-8bf6-4c57-bb80-49af49f1c645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:12.305504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-12236489-8bf6-4c57-bb80-49af49f1c645 None None] MFA Rules not processed for user `b4116bba47f74b399a01831a66e1a213`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:56:12.307398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-12236489-8bf6-4c57-bb80-49af49f1c645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:12.328226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986e11c-3a07-41c5-aa7c-d06edd3c28a5 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:12.330760 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d986e11c-3a07-41c5-aa7c-d06edd3c28a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:12.399439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1139/2258] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:56:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:56:12.413688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12236489-8bf6-4c57-bb80-49af49f1c645 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:12.415802 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-12236489-8bf6-4c57-bb80-49af49f1c645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:12.434884 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986e11c-3a07-41c5-aa7c-d06edd3c28a5 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:12.442571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74af38b-831a-4953-b437-ac74db435415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:12.443433 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74af38b-831a-4953-b437-ac74db435415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:12.443781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74af38b-831a-4953-b437-ac74db435415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:12.444275 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74af38b-831a-4953-b437-ac74db435415 admin admin] PATH_INFO: `/v3/projects/2ccac39e68324aaeb32d5eb06b8d685c` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:12.446267 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74af38b-831a-4953-b437-ac74db435415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ccac39e68324aaeb32d5eb06b8d685c)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:12.449169 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74af38b-831a-4953-b437-ac74db435415 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:12.503992 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12236489-8bf6-4c57-bb80-49af49f1c645 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:12.509478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155e951-49a9-47b8-b96b-92a8be6a48a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce5ec441b06408bbadd6fc040af0da5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:12.511232 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155e951-49a9-47b8-b96b-92a8be6a48a3 service glance] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:12.511671 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155e951-49a9-47b8-b96b-92a8be6a48a3 service glance] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:12.512141 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155e951-49a9-47b8-b96b-92a8be6a48a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:12.527991 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155e951-49a9-47b8-b96b-92a8be6a48a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:12.529750 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155e951-49a9-47b8-b96b-92a8be6a48a3 service glance] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:12.539176 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f74af38b-831a-4953-b437-ac74db435415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ccac39e68324aaeb32d5eb06b8d685c', 'request_id': 'req-f74af38b-831a-4953-b437-ac74db435415'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:12.542428 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1140/2259] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:56:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:56:12.556904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f74af38b-831a-4953-b437-ac74db435415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ccac39e68324aaeb32d5eb06b8d685c', 'request_id': 'req-f74af38b-831a-4953-b437-ac74db435415'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:12.565460 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f74af38b-831a-4953-b437-ac74db435415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ccac39e68324aaeb32d5eb06b8d685c', 'request_id': 'req-f74af38b-831a-4953-b437-ac74db435415'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:12.591598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f74af38b-831a-4953-b437-ac74db435415 admin admin] The token cache is being invalidate because project 2ccac39e68324aaeb32d5eb06b8d685c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:56:12.592021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f74af38b-831a-4953-b437-ac74db435415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:12.592710 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f74af38b-831a-4953-b437-ac74db435415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:12.599375 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155e951-49a9-47b8-b96b-92a8be6a48a3 service glance] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:12.599987 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1120/2260] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:56:12 2022] DELETE /identity/v3/projects/2ccac39e68324aaeb32d5eb06b8d685c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:56:12.613249 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155e951-49a9-47b8-b96b-92a8be6a48a3 service glance] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:12.633405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:12.698328 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be60957-d6a7-48b1-acdf-b63a952bf73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:12.699257 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be60957-d6a7-48b1-acdf-b63a952bf73c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:12.699849 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be60957-d6a7-48b1-acdf-b63a952bf73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:12.700379 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be60957-d6a7-48b1-acdf-b63a952bf73c admin admin] PATH_INFO: `/v3/users/2d34bd934b174453bc439af428948441` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:12.702564 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be60957-d6a7-48b1-acdf-b63a952bf73c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d34bd934b174453bc439af428948441)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:12.705541 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be60957-d6a7-48b1-acdf-b63a952bf73c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:12.777771 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9be60957-d6a7-48b1-acdf-b63a952bf73c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d34bd934b174453bc439af428948441'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:12.793903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9be60957-d6a7-48b1-acdf-b63a952bf73c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d34bd934b174453bc439af428948441'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:12.803981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9be60957-d6a7-48b1-acdf-b63a952bf73c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d34bd934b174453bc439af428948441'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:12.804949 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9be60957-d6a7-48b1-acdf-b63a952bf73c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d34bd934b174453bc439af428948441'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:12.817272 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9be60957-d6a7-48b1-acdf-b63a952bf73c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d34bd934b174453bc439af428948441'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:12.830962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1141/2261] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:56:12 2022] DELETE /identity/v3/users/2d34bd934b174453bc439af428948441 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:56:13.237558 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74af38b-831a-4953-b437-ac74db435415 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:13.239027 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-f74af38b-831a-4953-b437-ac74db435415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:13.255301 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74af38b-831a-4953-b437-ac74db435415 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:13.320230 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:13.332081 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebe412-f955-4f95-8ca3-37e024295a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:13.335785 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebe412-f955-4f95-8ca3-37e024295a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:13.336122 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebe412-f955-4f95-8ca3-37e024295a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:13.336531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebe412-f955-4f95-8ca3-37e024295a81 admin admin] PATH_INFO: `/v3/projects/22b1508f8a6d4dc5867d07508f766c30` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:13.337823 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebe412-f955-4f95-8ca3-37e024295a81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22b1508f8a6d4dc5867d07508f766c30)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:13.339798 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebe412-f955-4f95-8ca3-37e024295a81 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:13.392163 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a8ebe412-f955-4f95-8ca3-37e024295a81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22b1508f8a6d4dc5867d07508f766c30', 'request_id': 'req-a8ebe412-f955-4f95-8ca3-37e024295a81'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:13.401639 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a8ebe412-f955-4f95-8ca3-37e024295a81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22b1508f8a6d4dc5867d07508f766c30', 'request_id': 'req-a8ebe412-f955-4f95-8ca3-37e024295a81'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:13.408447 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a8ebe412-f955-4f95-8ca3-37e024295a81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22b1508f8a6d4dc5867d07508f766c30', 'request_id': 'req-a8ebe412-f955-4f95-8ca3-37e024295a81'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:13.421708 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a8ebe412-f955-4f95-8ca3-37e024295a81 admin admin] The token cache is being invalidate because project 22b1508f8a6d4dc5867d07508f766c30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:56:13.421942 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a8ebe412-f955-4f95-8ca3-37e024295a81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:13.422559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a8ebe412-f955-4f95-8ca3-37e024295a81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:13.425082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1121/2262] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:56:13 2022] DELETE /identity/v3/projects/22b1508f8a6d4dc5867d07508f766c30 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:56:13.428793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be60957-d6a7-48b1-acdf-b63a952bf73c admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:13.430218 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9be60957-d6a7-48b1-acdf-b63a952bf73c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:13.443663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be60957-d6a7-48b1-acdf-b63a952bf73c admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:13.514047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce81f65-341c-49cf-b588-adb7e33965b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:13.514688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce81f65-341c-49cf-b588-adb7e33965b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:13.514985 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce81f65-341c-49cf-b588-adb7e33965b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:13.515232 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce81f65-341c-49cf-b588-adb7e33965b8 admin admin] PATH_INFO: `/v3/users/16a2a7ec18e74aedbbd94d6d745fef07` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:13.516396 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce81f65-341c-49cf-b588-adb7e33965b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16a2a7ec18e74aedbbd94d6d745fef07)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:13.518545 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce81f65-341c-49cf-b588-adb7e33965b8 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:13.573318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:13.582885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-3ce81f65-341c-49cf-b588-adb7e33965b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16a2a7ec18e74aedbbd94d6d745fef07'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:13.604282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-3ce81f65-341c-49cf-b588-adb7e33965b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16a2a7ec18e74aedbbd94d6d745fef07'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:13.614966 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-3ce81f65-341c-49cf-b588-adb7e33965b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16a2a7ec18e74aedbbd94d6d745fef07'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:13.615697 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-3ce81f65-341c-49cf-b588-adb7e33965b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16a2a7ec18e74aedbbd94d6d745fef07'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:13.625818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-3ce81f65-341c-49cf-b588-adb7e33965b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16a2a7ec18e74aedbbd94d6d745fef07'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:13.637472 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1142/2263] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:56:13 2022] DELETE /identity/v3/users/16a2a7ec18e74aedbbd94d6d745fef07 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:56:13.987795 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebe412-f955-4f95-8ca3-37e024295a81 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:13.989391 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a8ebe412-f955-4f95-8ca3-37e024295a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:14.004964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebe412-f955-4f95-8ca3-37e024295a81 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:14.084893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6d5e2-4f02-482b-827a-e68b18c019a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:14.085541 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6d5e2-4f02-482b-827a-e68b18c019a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:14.085820 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6d5e2-4f02-482b-827a-e68b18c019a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:14.086088 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6d5e2-4f02-482b-827a-e68b18c019a7 admin admin] PATH_INFO: `/v3/projects/0112254eb831424bbce289c60151b9ae` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:14.087553 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6d5e2-4f02-482b-827a-e68b18c019a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0112254eb831424bbce289c60151b9ae)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:14.089492 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6d5e2-4f02-482b-827a-e68b18c019a7 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:14.148386 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-72a6d5e2-4f02-482b-827a-e68b18c019a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0112254eb831424bbce289c60151b9ae', 'request_id': 'req-72a6d5e2-4f02-482b-827a-e68b18c019a7'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:14.156799 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-72a6d5e2-4f02-482b-827a-e68b18c019a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0112254eb831424bbce289c60151b9ae', 'request_id': 'req-72a6d5e2-4f02-482b-827a-e68b18c019a7'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:14.166864 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-72a6d5e2-4f02-482b-827a-e68b18c019a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0112254eb831424bbce289c60151b9ae', 'request_id': 'req-72a6d5e2-4f02-482b-827a-e68b18c019a7'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:14.179511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-72a6d5e2-4f02-482b-827a-e68b18c019a7 admin admin] The token cache is being invalidate because project 0112254eb831424bbce289c60151b9ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:56:14.179832 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-72a6d5e2-4f02-482b-827a-e68b18c019a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:14.180499 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-72a6d5e2-4f02-482b-827a-e68b18c019a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:14.181998 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1122/2264] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:56:13 2022] DELETE /identity/v3/projects/0112254eb831424bbce289c60151b9ae => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:56:14.189962 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648d293-ada3-459b-be1c-5bb0baa833af None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:14.190535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648d293-ada3-459b-be1c-5bb0baa833af None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:14.190657 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648d293-ada3-459b-be1c-5bb0baa833af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:14.217157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-4648d293-ada3-459b-be1c-5bb0baa833af None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:56:14.218442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-4648d293-ada3-459b-be1c-5bb0baa833af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:14.288831 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1143/2265] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:56:14.296681 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee7486-feaf-4da7-ac35-6c9739c0aedd None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:14.297289 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee7486-feaf-4da7-ac35-6c9739c0aedd None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:14.297669 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee7486-feaf-4da7-ac35-6c9739c0aedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:14.323852 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-1fee7486-feaf-4da7-ac35-6c9739c0aedd None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:56:14.325312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1fee7486-feaf-4da7-ac35-6c9739c0aedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:14.334879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1123/2266] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:56:14.338550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648d293-ada3-459b-be1c-5bb0baa833af None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:14.349669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648d293-ada3-459b-be1c-5bb0baa833af None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:14.354280 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c7e87-3919-48ca-9135-b9f12bedbbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:14.354891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7e87-3919-48ca-9135-b9f12bedbbca admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:14.355170 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7e87-3919-48ca-9135-b9f12bedbbca admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:14.355441 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7e87-3919-48ca-9135-b9f12bedbbca admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:14.356119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c7e87-3919-48ca-9135-b9f12bedbbca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:56:14.356432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c7e87-3919-48ca-9135-b9f12bedbbca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:14.358023 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c7e87-3919-48ca-9135-b9f12bedbbca admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:14.374533 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1144/2267] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 21:56:14.375655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee7486-feaf-4da7-ac35-6c9739c0aedd None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:14.386622 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee7486-feaf-4da7-ac35-6c9739c0aedd None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:14.395108 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff2fd9-57c7-4de5-ab9b-7dec1aed64fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:14.395885 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff2fd9-57c7-4de5-ab9b-7dec1aed64fd admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:14.396236 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff2fd9-57c7-4de5-ab9b-7dec1aed64fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:14.396737 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff2fd9-57c7-4de5-ab9b-7dec1aed64fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:14.397462 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff2fd9-57c7-4de5-ab9b-7dec1aed64fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:14.399480 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff2fd9-57c7-4de5-ab9b-7dec1aed64fd admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:14.410781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:14.432563 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1124/2268] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:56:14 2022] POST /identity/v3/projects => generated 389 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:56:14.437250 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c7e87-3919-48ca-9135-b9f12bedbbca admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:14.448164 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c7e87-3919-48ca-9135-b9f12bedbbca admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:14.519540 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45622959-30fe-4898-8356-544fc5f2f055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:14.520265 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45622959-30fe-4898-8356-544fc5f2f055 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:14.520648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45622959-30fe-4898-8356-544fc5f2f055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:14.521025 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45622959-30fe-4898-8356-544fc5f2f055 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:14.522002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45622959-30fe-4898-8356-544fc5f2f055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:14.523809 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45622959-30fe-4898-8356-544fc5f2f055 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:14.547882 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1145/2269] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:56:14 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:56:14.552474 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff2fd9-57c7-4de5-ab9b-7dec1aed64fd admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:14.565620 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff2fd9-57c7-4de5-ab9b-7dec1aed64fd admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:14.570580 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a117cb4-68b7-4430-bee9-8b2cdd6a6be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:14.571266 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a117cb4-68b7-4430-bee9-8b2cdd6a6be9 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:14.571693 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a117cb4-68b7-4430-bee9-8b2cdd6a6be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:14.572016 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a117cb4-68b7-4430-bee9-8b2cdd6a6be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:14.572729 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a117cb4-68b7-4430-bee9-8b2cdd6a6be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:14.574390 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a117cb4-68b7-4430-bee9-8b2cdd6a6be9 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:14.587623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1125/2270] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:56:14.590400 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45622959-30fe-4898-8356-544fc5f2f055 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:14.605418 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45622959-30fe-4898-8356-544fc5f2f055 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:14.610192 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb7ff6-857b-42d9-985b-0e89817d6047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:14.611370 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb7ff6-857b-42d9-985b-0e89817d6047 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:14.611676 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb7ff6-857b-42d9-985b-0e89817d6047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:14.611949 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb7ff6-857b-42d9-985b-0e89817d6047 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:14.612594 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb7ff6-857b-42d9-985b-0e89817d6047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:14.614228 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb7ff6-857b-42d9-985b-0e89817d6047 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:14.628595 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1146/2271] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:56:14.633148 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a117cb4-68b7-4430-bee9-8b2cdd6a6be9 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:14.658527 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a117cb4-68b7-4430-bee9-8b2cdd6a6be9 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:14.668182 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50aeb7-6579-413c-9fa1-c3ba8ba72fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:14.668882 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50aeb7-6579-413c-9fa1-c3ba8ba72fa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:14.669167 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50aeb7-6579-413c-9fa1-c3ba8ba72fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:14.669445 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50aeb7-6579-413c-9fa1-c3ba8ba72fa9 admin admin] PATH_INFO: `/v3/projects/a0d7c901940c4a26bf67c324a0940643/users/4c7f1f32103c41c488b9393540bace16/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:14.686888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50aeb7-6579-413c-9fa1-c3ba8ba72fa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d7c901940c4a26bf67c324a0940643, user_id=4c7f1f32103c41c488b9393540bace16, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:14.689747 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50aeb7-6579-413c-9fa1-c3ba8ba72fa9 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:14.711848 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1126/2272] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:56:14 2022] PUT /identity/v3/projects/a0d7c901940c4a26bf67c324a0940643/users/4c7f1f32103c41c488b9393540bace16/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:56:14.724585 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38ebe5-15fb-4b8b-b093-230b1d296904 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:14.724927 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38ebe5-15fb-4b8b-b093-230b1d296904 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:14.725259 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38ebe5-15fb-4b8b-b093-230b1d296904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:14.756756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-5c38ebe5-15fb-4b8b-b093-230b1d296904 None None] MFA Rules not processed for user `4c7f1f32103c41c488b9393540bace16`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:56:14.758001 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-5c38ebe5-15fb-4b8b-b093-230b1d296904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:14.768123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:14.837961 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1147/2273] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:56:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:56:14.852257 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50aeb7-6579-413c-9fa1-c3ba8ba72fa9 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:14.854051 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8d50aeb7-6579-413c-9fa1-c3ba8ba72fa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:14.916841 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50aeb7-6579-413c-9fa1-c3ba8ba72fa9 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:14.920670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010e6ce-29be-4365-876e-b0f83596d4a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:14.921360 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010e6ce-29be-4365-876e-b0f83596d4a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:14.921726 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010e6ce-29be-4365-876e-b0f83596d4a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:14.922069 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010e6ce-29be-4365-876e-b0f83596d4a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:14.934184 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010e6ce-29be-4365-876e-b0f83596d4a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:14.935650 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010e6ce-29be-4365-876e-b0f83596d4a7 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:14.947096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1127/2274] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:56:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:56:15.504097 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1877f-ecbc-4acb-87b3-a1411eec6b56 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:15.504630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1877f-ecbc-4acb-87b3-a1411eec6b56 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:15.504630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1877f-ecbc-4acb-87b3-a1411eec6b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:15.523337 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-56d1877f-ecbc-4acb-87b3-a1411eec6b56 None None] MFA Rules not processed for user `933f34b82d5b4355844a57707de417a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:56:15.524565 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-56d1877f-ecbc-4acb-87b3-a1411eec6b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:15.577717 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1148/2275] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:56:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:56:16.059714 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010e6ce-29be-4365-876e-b0f83596d4a7 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:16.061171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d010e6ce-29be-4365-876e-b0f83596d4a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:16.118225 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010e6ce-29be-4365-876e-b0f83596d4a7 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:16.121911 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9414e6-5e26-4a10-b3f1-5a9b72180e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:16.122542 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9414e6-5e26-4a10-b3f1-5a9b72180e2b service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:16.122820 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9414e6-5e26-4a10-b3f1-5a9b72180e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:16.123095 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9414e6-5e26-4a10-b3f1-5a9b72180e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:16.124928 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-6a9414e6-5e26-4a10-b3f1-5a9b72180e2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:16.134531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9414e6-5e26-4a10-b3f1-5a9b72180e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:16.136269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9414e6-5e26-4a10-b3f1-5a9b72180e2b service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:16.144747 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1128/2276] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:56:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:56:16.972770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1877f-ecbc-4acb-87b3-a1411eec6b56 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:16.974177 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-56d1877f-ecbc-4acb-87b3-a1411eec6b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:17.044471 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1877f-ecbc-4acb-87b3-a1411eec6b56 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:17.049539 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314eb7b9-0cd3-43ac-86da-42b6de781225 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:17.050152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314eb7b9-0cd3-43ac-86da-42b6de781225 service placement] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:17.050427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314eb7b9-0cd3-43ac-86da-42b6de781225 service placement] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:17.050694 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314eb7b9-0cd3-43ac-86da-42b6de781225 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:17.052508 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-314eb7b9-0cd3-43ac-86da-42b6de781225 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:17.064892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314eb7b9-0cd3-43ac-86da-42b6de781225 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:17.066520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314eb7b9-0cd3-43ac-86da-42b6de781225 service placement] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:17.076795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1149/2277] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:56:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:56:18.072311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4dd938-b0f5-4c1b-951d-e49b245fbb47 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:18.072804 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4dd938-b0f5-4c1b-951d-e49b245fbb47 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:18.072914 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4dd938-b0f5-4c1b-951d-e49b245fbb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:18.092954 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-2c4dd938-b0f5-4c1b-951d-e49b245fbb47 None None] MFA Rules not processed for user `2346003afe74477496045e9bc72173d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:56:18.094162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2c4dd938-b0f5-4c1b-951d-e49b245fbb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:18.153109 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1129/2278] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Oct 21 21:56:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:56:18.166458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314eb7b9-0cd3-43ac-86da-42b6de781225 service placement] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:18.167902 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-314eb7b9-0cd3-43ac-86da-42b6de781225 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:18.232323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:18.257671 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314eb7b9-0cd3-43ac-86da-42b6de781225 service placement] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:18.263025 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03708fec-4ffe-4012-a210-08aec1610fb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ce720c8df014f9a8af062138aecd1b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:18.263671 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03708fec-4ffe-4012-a210-08aec1610fb9 service swift] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:18.263947 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03708fec-4ffe-4012-a210-08aec1610fb9 service swift] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:18.264221 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03708fec-4ffe-4012-a210-08aec1610fb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:18.273272 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03708fec-4ffe-4012-a210-08aec1610fb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:18.274876 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03708fec-4ffe-4012-a210-08aec1610fb9 service swift] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:18.284594 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1150/2279] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Oct 21 21:56:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 21:56:27.819405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4dd938-b0f5-4c1b-951d-e49b245fbb47 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:27.821197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2c4dd938-b0f5-4c1b-951d-e49b245fbb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:27.838338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:27.906604 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4dd938-b0f5-4c1b-951d-e49b245fbb47 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:27.932863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857365e-6241-4683-a6d9-a5a9b57733ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:27.933659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857365e-6241-4683-a6d9-a5a9b57733ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:27.934706 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857365e-6241-4683-a6d9-a5a9b57733ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:27.935154 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857365e-6241-4683-a6d9-a5a9b57733ce admin admin] PATH_INFO: `/v3/users/4921f7950f2441d0b2b4154ee269cb17` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:27.938571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c857365e-6241-4683-a6d9-a5a9b57733ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=4921f7950f2441d0b2b4154ee269cb17)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:27.941876 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c857365e-6241-4683-a6d9-a5a9b57733ce admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:28.009664 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c857365e-6241-4683-a6d9-a5a9b57733ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4921f7950f2441d0b2b4154ee269cb17'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:28.010667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c857365e-6241-4683-a6d9-a5a9b57733ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4921f7950f2441d0b2b4154ee269cb17'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:28.045727 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c857365e-6241-4683-a6d9-a5a9b57733ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4921f7950f2441d0b2b4154ee269cb17'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:28.069404 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c857365e-6241-4683-a6d9-a5a9b57733ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4921f7950f2441d0b2b4154ee269cb17'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:28.070035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-c857365e-6241-4683-a6d9-a5a9b57733ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4921f7950f2441d0b2b4154ee269cb17'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:28.093456 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1130/2280] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:56:27 2022] DELETE /identity/v3/users/4921f7950f2441d0b2b4154ee269cb17 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:56:28.098778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d7202-cf28-4d76-8677-a6a120f542f1 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:28.099355 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d7202-cf28-4d76-8677-a6a120f542f1 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:28.099355 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d7202-cf28-4d76-8677-a6a120f542f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:28.132473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-9a5d7202-cf28-4d76-8677-a6a120f542f1 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:56:28.136530 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9a5d7202-cf28-4d76-8677-a6a120f542f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:28.246078 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1151/2281] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:56:28.253516 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857365e-6241-4683-a6d9-a5a9b57733ce admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:28.255003 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c857365e-6241-4683-a6d9-a5a9b57733ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:28.369639 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857365e-6241-4683-a6d9-a5a9b57733ce admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:28.373461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94294ac2-843a-4029-b570-4fbac5b08ad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:28.374154 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94294ac2-843a-4029-b570-4fbac5b08ad7 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:28.374451 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94294ac2-843a-4029-b570-4fbac5b08ad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:28.375412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94294ac2-843a-4029-b570-4fbac5b08ad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:28.386309 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94294ac2-843a-4029-b570-4fbac5b08ad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:28.388354 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94294ac2-843a-4029-b570-4fbac5b08ad7 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:28.401990 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1131/2282] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:56:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:56:28.765148 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d7202-cf28-4d76-8677-a6a120f542f1 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:28.766627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9a5d7202-cf28-4d76-8677-a6a120f542f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:28.778771 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d7202-cf28-4d76-8677-a6a120f542f1 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:28.783411 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada3f33-51ef-4b5e-990e-7706d8daf30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:28.784144 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada3f33-51ef-4b5e-990e-7706d8daf30a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:28.784510 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada3f33-51ef-4b5e-990e-7706d8daf30a admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:28.784842 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada3f33-51ef-4b5e-990e-7706d8daf30a admin admin] PATH_INFO: `/v3/projects/9f8845452eec4010b351362acd1186e8` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:28.786014 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada3f33-51ef-4b5e-990e-7706d8daf30a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f8845452eec4010b351362acd1186e8)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:28.788033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada3f33-51ef-4b5e-990e-7706d8daf30a admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:28.833625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4ada3f33-51ef-4b5e-990e-7706d8daf30a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f8845452eec4010b351362acd1186e8', 'request_id': 'req-4ada3f33-51ef-4b5e-990e-7706d8daf30a'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:28.840372 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4ada3f33-51ef-4b5e-990e-7706d8daf30a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f8845452eec4010b351362acd1186e8', 'request_id': 'req-4ada3f33-51ef-4b5e-990e-7706d8daf30a'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:28.849406 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4ada3f33-51ef-4b5e-990e-7706d8daf30a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f8845452eec4010b351362acd1186e8', 'request_id': 'req-4ada3f33-51ef-4b5e-990e-7706d8daf30a'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:28.885773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4ada3f33-51ef-4b5e-990e-7706d8daf30a admin admin] The token cache is being invalidate because project 9f8845452eec4010b351362acd1186e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:56:28.886404 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4ada3f33-51ef-4b5e-990e-7706d8daf30a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:28.887425 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4ada3f33-51ef-4b5e-990e-7706d8daf30a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:28.889588 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1152/2283] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:56:28 2022] DELETE /identity/v3/projects/9f8845452eec4010b351362acd1186e8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:56:28.912835 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94294ac2-843a-4029-b570-4fbac5b08ad7 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:28.925083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94294ac2-843a-4029-b570-4fbac5b08ad7 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:28.949367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:28.981719 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597af59-4ebb-4cb9-968c-6af2f23b38b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:28.982258 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597af59-4ebb-4cb9-968c-6af2f23b38b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:28.982677 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597af59-4ebb-4cb9-968c-6af2f23b38b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:28.982952 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597af59-4ebb-4cb9-968c-6af2f23b38b5 admin admin] PATH_INFO: `/v3/users/54a658ece92d455e8ad44cfaba299ef6` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:28.984099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597af59-4ebb-4cb9-968c-6af2f23b38b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54a658ece92d455e8ad44cfaba299ef6)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:28.986551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597af59-4ebb-4cb9-968c-6af2f23b38b5 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:29.037912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-4597af59-4ebb-4cb9-968c-6af2f23b38b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54a658ece92d455e8ad44cfaba299ef6'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:29.038782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-4597af59-4ebb-4cb9-968c-6af2f23b38b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54a658ece92d455e8ad44cfaba299ef6'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:29.060984 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-4597af59-4ebb-4cb9-968c-6af2f23b38b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54a658ece92d455e8ad44cfaba299ef6'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:29.069116 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-4597af59-4ebb-4cb9-968c-6af2f23b38b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54a658ece92d455e8ad44cfaba299ef6'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:29.069784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-4597af59-4ebb-4cb9-968c-6af2f23b38b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54a658ece92d455e8ad44cfaba299ef6'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:29.082218 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1132/2284] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:56:28 2022] DELETE /identity/v3/users/54a658ece92d455e8ad44cfaba299ef6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:56:29.440333 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada3f33-51ef-4b5e-990e-7706d8daf30a admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:29.444477 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-4ada3f33-51ef-4b5e-990e-7706d8daf30a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:29.463178 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada3f33-51ef-4b5e-990e-7706d8daf30a admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:29.554128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:29.563701 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45584fe8-680d-40ac-ae55-44c451532574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:29.563701 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45584fe8-680d-40ac-ae55-44c451532574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:29.563701 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45584fe8-680d-40ac-ae55-44c451532574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:29.563701 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45584fe8-680d-40ac-ae55-44c451532574 admin admin] PATH_INFO: `/v3/projects/7232e296e47c47e4800411586586b035` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:29.564663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45584fe8-680d-40ac-ae55-44c451532574 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7232e296e47c47e4800411586586b035)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:29.567320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45584fe8-680d-40ac-ae55-44c451532574 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:29.633722 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-45584fe8-680d-40ac-ae55-44c451532574 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7232e296e47c47e4800411586586b035', 'request_id': 'req-45584fe8-680d-40ac-ae55-44c451532574'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:29.658767 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-45584fe8-680d-40ac-ae55-44c451532574 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7232e296e47c47e4800411586586b035', 'request_id': 'req-45584fe8-680d-40ac-ae55-44c451532574'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:29.667667 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-45584fe8-680d-40ac-ae55-44c451532574 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7232e296e47c47e4800411586586b035', 'request_id': 'req-45584fe8-680d-40ac-ae55-44c451532574'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:29.681071 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-45584fe8-680d-40ac-ae55-44c451532574 admin admin] The token cache is being invalidate because project 7232e296e47c47e4800411586586b035 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:56:29.681349 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-45584fe8-680d-40ac-ae55-44c451532574 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:29.682016 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-45584fe8-680d-40ac-ae55-44c451532574 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:56:29.683998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1153/2285] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:56:29 2022] DELETE /identity/v3/projects/7232e296e47c47e4800411586586b035 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:56:29.688886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1d0f4-1b24-4d35-966c-02309c383198 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:29.689406 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1d0f4-1b24-4d35-966c-02309c383198 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:29.689683 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1d0f4-1b24-4d35-966c-02309c383198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:29.725577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-f6b1d0f4-1b24-4d35-966c-02309c383198 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:56:29.726943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-f6b1d0f4-1b24-4d35-966c-02309c383198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:29.819724 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1133/2286] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:56:29.831855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af8a2d-f643-49e8-b513-dded445796bd None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:29.832148 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af8a2d-f643-49e8-b513-dded445796bd None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:29.832428 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af8a2d-f643-49e8-b513-dded445796bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:29.852805 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-51af8a2d-f643-49e8-b513-dded445796bd None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:56:29.854060 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-51af8a2d-f643-49e8-b513-dded445796bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:29.862783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1154/2287] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:56:29.869193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1d0f4-1b24-4d35-966c-02309c383198 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:29.897827 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1d0f4-1b24-4d35-966c-02309c383198 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:29.904421 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04428c68-280d-446b-9a37-704aa8e9e6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:29.915169 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04428c68-280d-446b-9a37-704aa8e9e6bf admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:29.915876 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04428c68-280d-446b-9a37-704aa8e9e6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:29.915876 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04428c68-280d-446b-9a37-704aa8e9e6bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:29.916445 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04428c68-280d-446b-9a37-704aa8e9e6bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:56:29.916754 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04428c68-280d-446b-9a37-704aa8e9e6bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:29.918387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04428c68-280d-446b-9a37-704aa8e9e6bf admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:29.933537 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1134/2288] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:56:29.947598 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af8a2d-f643-49e8-b513-dded445796bd None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:29.950554 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af8a2d-f643-49e8-b513-dded445796bd None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:29.956138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f346428-3933-40c7-8485-273bdf32d6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:29.956759 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f346428-3933-40c7-8485-273bdf32d6b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:29.957041 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f346428-3933-40c7-8485-273bdf32d6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:29.957313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f346428-3933-40c7-8485-273bdf32d6b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:29.957953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f346428-3933-40c7-8485-273bdf32d6b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:29.959911 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f346428-3933-40c7-8485-273bdf32d6b4 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:29.991690 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1155/2289] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:56:29 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:56:29.992994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04428c68-280d-446b-9a37-704aa8e9e6bf admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:30.003915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04428c68-280d-446b-9a37-704aa8e9e6bf admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:30.036818 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:30.064692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e48e2-0bc3-4156-b310-84fd3c8c79e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:30.065294 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e48e2-0bc3-4156-b310-84fd3c8c79e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:30.065572 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e48e2-0bc3-4156-b310-84fd3c8c79e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:30.065843 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e48e2-0bc3-4156-b310-84fd3c8c79e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:30.066457 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e48e2-0bc3-4156-b310-84fd3c8c79e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:30.068393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e48e2-0bc3-4156-b310-84fd3c8c79e6 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:30.092073 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1135/2290] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:56:29 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:56:30.093795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f346428-3933-40c7-8485-273bdf32d6b4 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:30.105305 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f346428-3933-40c7-8485-273bdf32d6b4 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:30.110121 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4a15a-a0e5-4ed2-9e01-e319470bfe24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:30.111272 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4a15a-a0e5-4ed2-9e01-e319470bfe24 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:30.111762 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4a15a-a0e5-4ed2-9e01-e319470bfe24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:30.112034 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4a15a-a0e5-4ed2-9e01-e319470bfe24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:30.112670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4a15a-a0e5-4ed2-9e01-e319470bfe24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:30.114274 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4a15a-a0e5-4ed2-9e01-e319470bfe24 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:30.128384 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1156/2291] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:56:30.129019 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e48e2-0bc3-4156-b310-84fd3c8c79e6 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:30.140233 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e48e2-0bc3-4156-b310-84fd3c8c79e6 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:30.145509 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19378e79-b91b-4755-8816-c55aaa1fab2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:30.146129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19378e79-b91b-4755-8816-c55aaa1fab2b admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:30.146409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19378e79-b91b-4755-8816-c55aaa1fab2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:30.146678 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19378e79-b91b-4755-8816-c55aaa1fab2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:30.147320 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19378e79-b91b-4755-8816-c55aaa1fab2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:30.148964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19378e79-b91b-4755-8816-c55aaa1fab2b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:30.161665 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1136/2292] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:56:30.165766 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4a15a-a0e5-4ed2-9e01-e319470bfe24 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:30.176680 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4a15a-a0e5-4ed2-9e01-e319470bfe24 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:30.184369 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ddbe9-662b-4e96-bb36-70341779345c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:30.185617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ddbe9-662b-4e96-bb36-70341779345c admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:30.186143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ddbe9-662b-4e96-bb36-70341779345c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:30.186657 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ddbe9-662b-4e96-bb36-70341779345c admin admin] PATH_INFO: `/v3/projects/7afac7d15c564888b4f67b9ec57d35c6/users/3c118cb49ae04ace966b0c753b5f6665/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:30.206750 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ddbe9-662b-4e96-bb36-70341779345c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7afac7d15c564888b4f67b9ec57d35c6, user_id=3c118cb49ae04ace966b0c753b5f6665, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:30.209751 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ddbe9-662b-4e96-bb36-70341779345c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:30.234427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1157/2293] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:56:30 2022] PUT /identity/v3/projects/7afac7d15c564888b4f67b9ec57d35c6/users/3c118cb49ae04ace966b0c753b5f6665/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:56:30.244664 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6057c13-3dca-4680-b1a2-b31dbe17d40d None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:30.244983 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6057c13-3dca-4680-b1a2-b31dbe17d40d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:30.245214 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6057c13-3dca-4680-b1a2-b31dbe17d40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:30.300236 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-e6057c13-3dca-4680-b1a2-b31dbe17d40d None None] MFA Rules not processed for user `3c118cb49ae04ace966b0c753b5f6665`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:56:30.301473 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-e6057c13-3dca-4680-b1a2-b31dbe17d40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:30.381011 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1137/2294] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:56:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:56:30.381815 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ddbe9-662b-4e96-bb36-70341779345c admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:30.446043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ddbe9-662b-4e96-bb36-70341779345c admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:30.450679 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25916e39-fcf3-4fb2-9ba9-ba1709145056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:30.451287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25916e39-fcf3-4fb2-9ba9-ba1709145056 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:30.451577 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25916e39-fcf3-4fb2-9ba9-ba1709145056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:30.451891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25916e39-fcf3-4fb2-9ba9-ba1709145056 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:30.452557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25916e39-fcf3-4fb2-9ba9-ba1709145056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:30.455168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25916e39-fcf3-4fb2-9ba9-ba1709145056 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:30.488756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1158/2295] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:56:30 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:56:30.492839 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6057c13-3dca-4680-b1a2-b31dbe17d40d None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:30.504039 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6057c13-3dca-4680-b1a2-b31dbe17d40d None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:30.581427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310fb1f-9e7c-4bfb-b8e3-ccb5869c3f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:30.582101 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310fb1f-9e7c-4bfb-b8e3-ccb5869c3f54 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:30.582389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310fb1f-9e7c-4bfb-b8e3-ccb5869c3f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:30.582665 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310fb1f-9e7c-4bfb-b8e3-ccb5869c3f54 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:30.583313 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310fb1f-9e7c-4bfb-b8e3-ccb5869c3f54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:30.585919 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310fb1f-9e7c-4bfb-b8e3-ccb5869c3f54 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:30.615753 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1138/2296] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:56:30 2022] POST /identity/v3/users => generated 297 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:56:30.617039 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25916e39-fcf3-4fb2-9ba9-ba1709145056 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:30.628990 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25916e39-fcf3-4fb2-9ba9-ba1709145056 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:30.634112 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc5460-9229-4f24-9db5-869997732805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:30.634746 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc5460-9229-4f24-9db5-869997732805 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:30.635467 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc5460-9229-4f24-9db5-869997732805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:30.635836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc5460-9229-4f24-9db5-869997732805 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:30.636549 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc5460-9229-4f24-9db5-869997732805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:30.638241 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc5460-9229-4f24-9db5-869997732805 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:30.640819 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:30.652634 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1159/2297] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:56:30.657089 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310fb1f-9e7c-4bfb-b8e3-ccb5869c3f54 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:30.667038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310fb1f-9e7c-4bfb-b8e3-ccb5869c3f54 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:30.671563 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc6a1c-21b0-4a11-b6a7-776d4e5afba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:30.672242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc6a1c-21b0-4a11-b6a7-776d4e5afba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:30.672546 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc6a1c-21b0-4a11-b6a7-776d4e5afba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:30.672819 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc6a1c-21b0-4a11-b6a7-776d4e5afba7 admin admin] PATH_INFO: `/v3/projects/5d1071b729bb4c59af9d2c7f73160dad/users/10c8f7336e4547a9bc189e32431e6e86/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:30.689551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc6a1c-21b0-4a11-b6a7-776d4e5afba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d1071b729bb4c59af9d2c7f73160dad, user_id=10c8f7336e4547a9bc189e32431e6e86, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:30.692401 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc6a1c-21b0-4a11-b6a7-776d4e5afba7 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:30.707249 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1139/2298] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:56:30 2022] PUT /identity/v3/projects/5d1071b729bb4c59af9d2c7f73160dad/users/10c8f7336e4547a9bc189e32431e6e86/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:56:30.711783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc5460-9229-4f24-9db5-869997732805 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:30.722340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc5460-9229-4f24-9db5-869997732805 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:30.780736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e17ebf-577b-4b75-990c-fce80732ef0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:30.781350 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e17ebf-577b-4b75-990c-fce80732ef0a admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:30.781633 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e17ebf-577b-4b75-990c-fce80732ef0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:30.781935 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e17ebf-577b-4b75-990c-fce80732ef0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:30.782572 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e17ebf-577b-4b75-990c-fce80732ef0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:30.784250 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e17ebf-577b-4b75-990c-fce80732ef0a admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:30.794394 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1160/2299] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:56:30.798336 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc6a1c-21b0-4a11-b6a7-776d4e5afba7 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:30.808925 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc6a1c-21b0-4a11-b6a7-776d4e5afba7 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:30.814635 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591113e-83e2-4d87-8c4a-ab6cd3844590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:30.815325 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591113e-83e2-4d87-8c4a-ab6cd3844590 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:30.815647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591113e-83e2-4d87-8c4a-ab6cd3844590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:30.815972 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591113e-83e2-4d87-8c4a-ab6cd3844590 admin admin] PATH_INFO: `/v3/projects/5d1071b729bb4c59af9d2c7f73160dad/users/10c8f7336e4547a9bc189e32431e6e86/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:30.817909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591113e-83e2-4d87-8c4a-ab6cd3844590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d1071b729bb4c59af9d2c7f73160dad, user_id=10c8f7336e4547a9bc189e32431e6e86, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:30.820583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591113e-83e2-4d87-8c4a-ab6cd3844590 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:30.822908 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:30.833992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1140/2300] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:56:30 2022] PUT /identity/v3/projects/5d1071b729bb4c59af9d2c7f73160dad/users/10c8f7336e4547a9bc189e32431e6e86/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:56:30.844298 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d1dc7-dd57-4d8c-aeca-8785111bfa47 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:30.844615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d1dc7-dd57-4d8c-aeca-8785111bfa47 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:30.844890 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d1dc7-dd57-4d8c-aeca-8785111bfa47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:30.873224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-e53d1dc7-dd57-4d8c-aeca-8785111bfa47 None None] MFA Rules not processed for user `10c8f7336e4547a9bc189e32431e6e86`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:56:30.874439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e53d1dc7-dd57-4d8c-aeca-8785111bfa47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:30.956145 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1161/2301] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:56:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:56:30.967210 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591113e-83e2-4d87-8c4a-ab6cd3844590 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:30.968981 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c591113e-83e2-4d87-8c4a-ab6cd3844590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:31.069133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591113e-83e2-4d87-8c4a-ab6cd3844590 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:31.072791 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce688630-6b40-4ca1-a4c6-62b2386f46a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:31.073410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce688630-6b40-4ca1-a4c6-62b2386f46a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:31.075826 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce688630-6b40-4ca1-a4c6-62b2386f46a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:31.076368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce688630-6b40-4ca1-a4c6-62b2386f46a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:31.088161 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce688630-6b40-4ca1-a4c6-62b2386f46a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:31.089721 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce688630-6b40-4ca1-a4c6-62b2386f46a1 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:31.099387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1141/2302] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:56:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:56:31.179044 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d1dc7-dd57-4d8c-aeca-8785111bfa47 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:31.191299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d1dc7-dd57-4d8c-aeca-8785111bfa47 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:31.197496 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2d476-6b6c-46e3-839d-f48bea1b6366 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:31.198282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2d476-6b6c-46e3-839d-f48bea1b6366 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:31.198594 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2d476-6b6c-46e3-839d-f48bea1b6366 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:31.198849 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2d476-6b6c-46e3-839d-f48bea1b6366 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:31.276008 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2d476-6b6c-46e3-839d-f48bea1b6366 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:31.277652 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2d476-6b6c-46e3-839d-f48bea1b6366 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:31.292359 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1162/2303] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:56:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:56:40.869829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce688630-6b40-4ca1-a4c6-62b2386f46a1 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:56:40.871256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ce688630-6b40-4ca1-a4c6-62b2386f46a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:40.958706 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce688630-6b40-4ca1-a4c6-62b2386f46a1 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:56:40.964341 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11271384-664d-4f71-9daa-1a13d6f5f214 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:56:40.964953 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11271384-664d-4f71-9daa-1a13d6f5f214 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:56:40.965227 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11271384-664d-4f71-9daa-1a13d6f5f214 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:56:40.965495 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11271384-664d-4f71-9daa-1a13d6f5f214 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:56:40.967841 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-11271384-664d-4f71-9daa-1a13d6f5f214 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:56:40.981291 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11271384-664d-4f71-9daa-1a13d6f5f214 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:56:40.982771 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11271384-664d-4f71-9daa-1a13d6f5f214 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:56:40.996471 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1142/2304] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:56:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:56:41.285987 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=92 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:56:50.991433 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:INSERT=1,keystone:SELECT=62 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:01.021959 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2d476-6b6c-46e3-839d-f48bea1b6366 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:01.082526 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2d476-6b6c-46e3-839d-f48bea1b6366 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:01.087087 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d16ce5-8caf-4ff5-bc11-7316a7073b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:01.087751 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d16ce5-8caf-4ff5-bc11-7316a7073b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:01.088037 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d16ce5-8caf-4ff5-bc11-7316a7073b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:01.088306 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d16ce5-8caf-4ff5-bc11-7316a7073b34 admin admin] PATH_INFO: `/v3/users/3c118cb49ae04ace966b0c753b5f6665` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:01.089417 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d16ce5-8caf-4ff5-bc11-7316a7073b34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c118cb49ae04ace966b0c753b5f6665)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:01.091342 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d16ce5-8caf-4ff5-bc11-7316a7073b34 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:01.135952 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-20d16ce5-8caf-4ff5-bc11-7316a7073b34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c118cb49ae04ace966b0c753b5f6665'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.148222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-20d16ce5-8caf-4ff5-bc11-7316a7073b34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c118cb49ae04ace966b0c753b5f6665'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.155209 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-20d16ce5-8caf-4ff5-bc11-7316a7073b34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c118cb49ae04ace966b0c753b5f6665'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.155859 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-20d16ce5-8caf-4ff5-bc11-7316a7073b34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c118cb49ae04ace966b0c753b5f6665'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.163241 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-20d16ce5-8caf-4ff5-bc11-7316a7073b34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c118cb49ae04ace966b0c753b5f6665'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.169014 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1163/2305] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:57:01 2022] DELETE /identity/v3/users/3c118cb49ae04ace966b0c753b5f6665 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:01.173492 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea584f-e930-40a6-addd-40eb3a8a5f43 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:01.173944 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea584f-e930-40a6-addd-40eb3a8a5f43 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:01.174090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea584f-e930-40a6-addd-40eb3a8a5f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:01.192514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-35ea584f-e930-40a6-addd-40eb3a8a5f43 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:01.193684 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-35ea584f-e930-40a6-addd-40eb3a8a5f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:01.251924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1143/2306] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:01.262286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d16ce5-8caf-4ff5-bc11-7316a7073b34 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:01.263730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-20d16ce5-8caf-4ff5-bc11-7316a7073b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:01.319136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d16ce5-8caf-4ff5-bc11-7316a7073b34 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:01.322797 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee0572-1b46-48fc-8bdf-372ea4945aa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:01.323402 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee0572-1b46-48fc-8bdf-372ea4945aa3 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:01.323733 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee0572-1b46-48fc-8bdf-372ea4945aa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:01.324015 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee0572-1b46-48fc-8bdf-372ea4945aa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:01.333559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee0572-1b46-48fc-8bdf-372ea4945aa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:01.334999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee0572-1b46-48fc-8bdf-372ea4945aa3 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:01.345049 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1164/2307] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:57:01.359273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea584f-e930-40a6-addd-40eb3a8a5f43 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:01.360754 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-35ea584f-e930-40a6-addd-40eb3a8a5f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:01.375984 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea584f-e930-40a6-addd-40eb3a8a5f43 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:01.381795 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43883383-82de-4f1c-a8ec-01197de403a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:01.382465 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43883383-82de-4f1c-a8ec-01197de403a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:01.382745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43883383-82de-4f1c-a8ec-01197de403a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:01.383017 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43883383-82de-4f1c-a8ec-01197de403a8 admin admin] PATH_INFO: `/v3/users/4c7f1f32103c41c488b9393540bace16` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:01.384192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43883383-82de-4f1c-a8ec-01197de403a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c7f1f32103c41c488b9393540bace16)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:01.386467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43883383-82de-4f1c-a8ec-01197de403a8 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:01.434868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-43883383-82de-4f1c-a8ec-01197de403a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c7f1f32103c41c488b9393540bace16'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.435654 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-43883383-82de-4f1c-a8ec-01197de403a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c7f1f32103c41c488b9393540bace16'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.448584 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-43883383-82de-4f1c-a8ec-01197de403a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c7f1f32103c41c488b9393540bace16'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.456278 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-43883383-82de-4f1c-a8ec-01197de403a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c7f1f32103c41c488b9393540bace16'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.456904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-43883383-82de-4f1c-a8ec-01197de403a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c7f1f32103c41c488b9393540bace16'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.473731 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1144/2308] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:57:01 2022] DELETE /identity/v3/users/4c7f1f32103c41c488b9393540bace16 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:01.478173 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747236f-25cc-4ffd-830d-c88929a2ced4 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:01.478440 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747236f-25cc-4ffd-830d-c88929a2ced4 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:01.478715 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747236f-25cc-4ffd-830d-c88929a2ced4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:01.502853 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-8747236f-25cc-4ffd-830d-c88929a2ced4 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:01.504167 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8747236f-25cc-4ffd-830d-c88929a2ced4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:01.525746 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:01.574793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1165/2309] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:57:01.590868 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43883383-82de-4f1c-a8ec-01197de403a8 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:01.592846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-43883383-82de-4f1c-a8ec-01197de403a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:01.656997 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747236f-25cc-4ffd-830d-c88929a2ced4 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:01.659033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8747236f-25cc-4ffd-830d-c88929a2ced4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:01.661320 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43883383-82de-4f1c-a8ec-01197de403a8 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:01.665200 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26de45-526c-4696-ac2f-e1aedd0d239e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:01.665933 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26de45-526c-4696-ac2f-e1aedd0d239e service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:01.666380 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26de45-526c-4696-ac2f-e1aedd0d239e service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:01.666653 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26de45-526c-4696-ac2f-e1aedd0d239e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:01.674217 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747236f-25cc-4ffd-830d-c88929a2ced4 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:01.676388 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26de45-526c-4696-ac2f-e1aedd0d239e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:01.678798 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26de45-526c-4696-ac2f-e1aedd0d239e service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:01.679025 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:01.680121 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:01.680412 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:01.680691 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a admin admin] PATH_INFO: `/v3/projects/7afac7d15c564888b4f67b9ec57d35c6` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:01.681848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7afac7d15c564888b4f67b9ec57d35c6)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:01.684270 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:01.703612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1145/2310] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:57:01.760128 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7afac7d15c564888b4f67b9ec57d35c6', 'request_id': 'req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.767492 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7afac7d15c564888b4f67b9ec57d35c6', 'request_id': 'req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.780080 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7afac7d15c564888b4f67b9ec57d35c6', 'request_id': 'req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.792250 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a admin admin] The token cache is being invalidate because project 7afac7d15c564888b4f67b9ec57d35c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:57:01.792892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.793587 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.795191 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1166/2311] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:57:01 2022] DELETE /identity/v3/projects/7afac7d15c564888b4f67b9ec57d35c6 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:57:01.800295 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26de45-526c-4696-ac2f-e1aedd0d239e service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:01.810285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26de45-526c-4696-ac2f-e1aedd0d239e service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:01.822895 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:01.881999 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b810ca-f953-4884-8244-7216ec8783c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:01.882550 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b810ca-f953-4884-8244-7216ec8783c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:01.882837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b810ca-f953-4884-8244-7216ec8783c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:01.883110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b810ca-f953-4884-8244-7216ec8783c5 admin admin] PATH_INFO: `/v3/users/10c8f7336e4547a9bc189e32431e6e86` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:01.884309 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b810ca-f953-4884-8244-7216ec8783c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10c8f7336e4547a9bc189e32431e6e86)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:01.886527 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b810ca-f953-4884-8244-7216ec8783c5 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:01.941378 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-00b810ca-f953-4884-8244-7216ec8783c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10c8f7336e4547a9bc189e32431e6e86'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.942112 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-00b810ca-f953-4884-8244-7216ec8783c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10c8f7336e4547a9bc189e32431e6e86'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.954643 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-00b810ca-f953-4884-8244-7216ec8783c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10c8f7336e4547a9bc189e32431e6e86'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.962346 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-00b810ca-f953-4884-8244-7216ec8783c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10c8f7336e4547a9bc189e32431e6e86'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.963268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-00b810ca-f953-4884-8244-7216ec8783c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10c8f7336e4547a9bc189e32431e6e86'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:01.977032 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1146/2312] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:57:01 2022] DELETE /identity/v3/users/10c8f7336e4547a9bc189e32431e6e86 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:02.045907 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:02.047808 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:02.062488 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c4bab-c8bb-4dba-9471-80c03e556f7a admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:02.125899 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01372bb2-411b-4a9c-8d91-7e2711493e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:02.126533 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01372bb2-411b-4a9c-8d91-7e2711493e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:02.126821 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01372bb2-411b-4a9c-8d91-7e2711493e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:02.127121 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01372bb2-411b-4a9c-8d91-7e2711493e72 admin admin] PATH_INFO: `/v3/projects/a0d7c901940c4a26bf67c324a0940643` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:02.128428 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01372bb2-411b-4a9c-8d91-7e2711493e72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0d7c901940c4a26bf67c324a0940643)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:02.130352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01372bb2-411b-4a9c-8d91-7e2711493e72 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:02.180662 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:02.182050 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-01372bb2-411b-4a9c-8d91-7e2711493e72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0d7c901940c4a26bf67c324a0940643', 'request_id': 'req-01372bb2-411b-4a9c-8d91-7e2711493e72'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:02.189888 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-01372bb2-411b-4a9c-8d91-7e2711493e72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0d7c901940c4a26bf67c324a0940643', 'request_id': 'req-01372bb2-411b-4a9c-8d91-7e2711493e72'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:02.199158 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-01372bb2-411b-4a9c-8d91-7e2711493e72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0d7c901940c4a26bf67c324a0940643', 'request_id': 'req-01372bb2-411b-4a9c-8d91-7e2711493e72'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:02.211329 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-01372bb2-411b-4a9c-8d91-7e2711493e72 admin admin] The token cache is being invalidate because project a0d7c901940c4a26bf67c324a0940643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:57:02.211787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-01372bb2-411b-4a9c-8d91-7e2711493e72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:02.212518 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-01372bb2-411b-4a9c-8d91-7e2711493e72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:02.214329 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1167/2313] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:57:02 2022] DELETE /identity/v3/projects/a0d7c901940c4a26bf67c324a0940643 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:02.300424 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b810ca-f953-4884-8244-7216ec8783c5 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:02.303002 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-00b810ca-f953-4884-8244-7216ec8783c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:02.319536 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b810ca-f953-4884-8244-7216ec8783c5 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:02.382629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a6f8d-d883-42d0-8114-5d877578421a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:02.383275 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a6f8d-d883-42d0-8114-5d877578421a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:02.383552 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a6f8d-d883-42d0-8114-5d877578421a admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:02.383856 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a6f8d-d883-42d0-8114-5d877578421a admin admin] PATH_INFO: `/v3/projects/5d1071b729bb4c59af9d2c7f73160dad` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:02.385267 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a6f8d-d883-42d0-8114-5d877578421a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d1071b729bb4c59af9d2c7f73160dad)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:02.387220 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a6f8d-d883-42d0-8114-5d877578421a admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:02.432134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:02.443349 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a60a6f8d-d883-42d0-8114-5d877578421a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d1071b729bb4c59af9d2c7f73160dad', 'request_id': 'req-a60a6f8d-d883-42d0-8114-5d877578421a'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:02.451749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a60a6f8d-d883-42d0-8114-5d877578421a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d1071b729bb4c59af9d2c7f73160dad', 'request_id': 'req-a60a6f8d-d883-42d0-8114-5d877578421a'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:02.458347 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a60a6f8d-d883-42d0-8114-5d877578421a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d1071b729bb4c59af9d2c7f73160dad', 'request_id': 'req-a60a6f8d-d883-42d0-8114-5d877578421a'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:02.469709 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a60a6f8d-d883-42d0-8114-5d877578421a admin admin] The token cache is being invalidate because project 5d1071b729bb4c59af9d2c7f73160dad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:57:02.469994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a60a6f8d-d883-42d0-8114-5d877578421a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:02.470651 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a60a6f8d-d883-42d0-8114-5d877578421a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:02.472696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1147/2314] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:57:02 2022] DELETE /identity/v3/projects/5d1071b729bb4c59af9d2c7f73160dad => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:02.477519 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649f50f-c9a6-42ab-97d6-d0e85fd98503 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:02.477808 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649f50f-c9a6-42ab-97d6-d0e85fd98503 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:02.478086 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649f50f-c9a6-42ab-97d6-d0e85fd98503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:02.497311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-b649f50f-c9a6-42ab-97d6-d0e85fd98503 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:02.498515 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b649f50f-c9a6-42ab-97d6-d0e85fd98503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:02.555950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1168/2315] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:02.563716 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f5218-6d75-48f5-bdd2-2ddc9b212e77 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:02.564027 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f5218-6d75-48f5-bdd2-2ddc9b212e77 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:02.564303 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f5218-6d75-48f5-bdd2-2ddc9b212e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:02.584095 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-927f5218-6d75-48f5-bdd2-2ddc9b212e77 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:02.585310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-927f5218-6d75-48f5-bdd2-2ddc9b212e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:02.593048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1148/2316] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:57:02.597508 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649f50f-c9a6-42ab-97d6-d0e85fd98503 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:02.606945 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649f50f-c9a6-42ab-97d6-d0e85fd98503 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:02.611479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ca4c5-4e23-425e-bc56-08c337ef7967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:02.612113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ca4c5-4e23-425e-bc56-08c337ef7967 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:02.612389 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ca4c5-4e23-425e-bc56-08c337ef7967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:02.612659 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ca4c5-4e23-425e-bc56-08c337ef7967 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:02.613295 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ca4c5-4e23-425e-bc56-08c337ef7967 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:57:02.613600 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ca4c5-4e23-425e-bc56-08c337ef7967 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:02.615193 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ca4c5-4e23-425e-bc56-08c337ef7967 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:02.626336 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1169/2317] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:57:02.630552 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f5218-6d75-48f5-bdd2-2ddc9b212e77 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:02.640044 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f5218-6d75-48f5-bdd2-2ddc9b212e77 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:02.645167 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c285694-6d91-44c4-9634-bcee7f4f8d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:02.645755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c285694-6d91-44c4-9634-bcee7f4f8d76 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:02.646035 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c285694-6d91-44c4-9634-bcee7f4f8d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:02.646309 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c285694-6d91-44c4-9634-bcee7f4f8d76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:02.646929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c285694-6d91-44c4-9634-bcee7f4f8d76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:02.648873 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c285694-6d91-44c4-9634-bcee7f4f8d76 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:02.676049 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1149/2318] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:57:02 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:57:02.680210 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ca4c5-4e23-425e-bc56-08c337ef7967 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:02.690400 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ca4c5-4e23-425e-bc56-08c337ef7967 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:02.756750 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea4add-c372-407f-95f7-b510af45c81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:02.757411 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4add-c372-407f-95f7-b510af45c81a admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:02.757985 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4add-c372-407f-95f7-b510af45c81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:02.757985 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4add-c372-407f-95f7-b510af45c81a admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:02.758694 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea4add-c372-407f-95f7-b510af45c81a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:02.760984 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea4add-c372-407f-95f7-b510af45c81a admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:02.786727 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1170/2319] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:57:02 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:57:02.790863 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c285694-6d91-44c4-9634-bcee7f4f8d76 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:02.801702 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c285694-6d91-44c4-9634-bcee7f4f8d76 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:02.806594 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8bc06-836d-4186-a6a6-d63abe1945ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:02.807295 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8bc06-836d-4186-a6a6-d63abe1945ce admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:02.807599 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8bc06-836d-4186-a6a6-d63abe1945ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:02.807884 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8bc06-836d-4186-a6a6-d63abe1945ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:02.808525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8bc06-836d-4186-a6a6-d63abe1945ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:02.810116 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8bc06-836d-4186-a6a6-d63abe1945ce admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:02.824527 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1150/2320] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:57:02.825597 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea4add-c372-407f-95f7-b510af45c81a admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:02.836129 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea4add-c372-407f-95f7-b510af45c81a admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:02.840985 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4251b-6737-4648-a4fd-1db6138afbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:02.841714 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4251b-6737-4648-a4fd-1db6138afbcd admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:02.841995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4251b-6737-4648-a4fd-1db6138afbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:02.842262 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4251b-6737-4648-a4fd-1db6138afbcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:02.842895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4251b-6737-4648-a4fd-1db6138afbcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:02.844721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4251b-6737-4648-a4fd-1db6138afbcd admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:02.855945 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1171/2321] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:57:02.859837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8bc06-836d-4186-a6a6-d63abe1945ce admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:02.869893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8bc06-836d-4186-a6a6-d63abe1945ce admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:02.874979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7f6bc-4659-4490-b8ef-eee0814cd179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:02.875653 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7f6bc-4659-4490-b8ef-eee0814cd179 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:02.875935 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7f6bc-4659-4490-b8ef-eee0814cd179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:02.876211 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7f6bc-4659-4490-b8ef-eee0814cd179 admin admin] PATH_INFO: `/v3/projects/61b1bfaf21e7460c92ac23b78f923945/users/9e4755af6a1440828d56173bd17a9803/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:02.891988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7f6bc-4659-4490-b8ef-eee0814cd179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61b1bfaf21e7460c92ac23b78f923945, user_id=9e4755af6a1440828d56173bd17a9803, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:02.895258 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7f6bc-4659-4490-b8ef-eee0814cd179 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:02.908282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1151/2322] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:57:02 2022] PUT /identity/v3/projects/61b1bfaf21e7460c92ac23b78f923945/users/9e4755af6a1440828d56173bd17a9803/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:57:02.918142 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fca17f-c8c5-4bed-9fc3-8db0eaf7f3f3 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:02.918524 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fca17f-c8c5-4bed-9fc3-8db0eaf7f3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:02.918863 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fca17f-c8c5-4bed-9fc3-8db0eaf7f3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:02.955010 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-37fca17f-c8c5-4bed-9fc3-8db0eaf7f3f3 None None] MFA Rules not processed for user `9e4755af6a1440828d56173bd17a9803`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:02.956847 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-37fca17f-c8c5-4bed-9fc3-8db0eaf7f3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:02.998937 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:03.036803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1172/2323] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:57:03.041455 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7f6bc-4659-4490-b8ef-eee0814cd179 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:03.102704 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7f6bc-4659-4490-b8ef-eee0814cd179 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:03.107357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958f13e-3648-4386-b7ef-164c5b187266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:03.107992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958f13e-3648-4386-b7ef-164c5b187266 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:03.108274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958f13e-3648-4386-b7ef-164c5b187266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:03.108551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958f13e-3648-4386-b7ef-164c5b187266 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:03.109180 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958f13e-3648-4386-b7ef-164c5b187266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:03.111078 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958f13e-3648-4386-b7ef-164c5b187266 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:03.142305 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1152/2324] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:57:03 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:57:03.143594 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fca17f-c8c5-4bed-9fc3-8db0eaf7f3f3 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:03.154316 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fca17f-c8c5-4bed-9fc3-8db0eaf7f3f3 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:03.211043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da66656-1efe-4055-81b6-9990f06f5134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:03.211623 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da66656-1efe-4055-81b6-9990f06f5134 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:03.211903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da66656-1efe-4055-81b6-9990f06f5134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:03.212174 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da66656-1efe-4055-81b6-9990f06f5134 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:03.212796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da66656-1efe-4055-81b6-9990f06f5134 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:03.214778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da66656-1efe-4055-81b6-9990f06f5134 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:03.239825 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1173/2325] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:57:03 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:57:03.241031 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958f13e-3648-4386-b7ef-164c5b187266 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:03.251562 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958f13e-3648-4386-b7ef-164c5b187266 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:03.262135 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656e993-871d-4da9-b12f-03856044f0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:03.262745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656e993-871d-4da9-b12f-03856044f0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:03.263024 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656e993-871d-4da9-b12f-03856044f0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:03.263292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656e993-871d-4da9-b12f-03856044f0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:03.263956 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656e993-871d-4da9-b12f-03856044f0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:03.265606 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656e993-871d-4da9-b12f-03856044f0e6 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:03.276326 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1153/2326] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:57:03.280531 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da66656-1efe-4055-81b6-9990f06f5134 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:03.290769 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da66656-1efe-4055-81b6-9990f06f5134 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:03.295420 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37758a9a-365b-42cc-a7f8-da0007134c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:03.296181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37758a9a-365b-42cc-a7f8-da0007134c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:03.296463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37758a9a-365b-42cc-a7f8-da0007134c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:03.296739 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37758a9a-365b-42cc-a7f8-da0007134c07 admin admin] PATH_INFO: `/v3/projects/35c132ce49124f0e8866510cf8f5e890/users/b8b9a024c3b942a6a188818f3e6c055f/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:03.312522 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37758a9a-365b-42cc-a7f8-da0007134c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c132ce49124f0e8866510cf8f5e890, user_id=b8b9a024c3b942a6a188818f3e6c055f, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:03.315811 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37758a9a-365b-42cc-a7f8-da0007134c07 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:03.329094 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1174/2327] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:57:03 2022] PUT /identity/v3/projects/35c132ce49124f0e8866510cf8f5e890/users/b8b9a024c3b942a6a188818f3e6c055f/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:57:03.333191 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656e993-871d-4da9-b12f-03856044f0e6 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:03.343412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656e993-871d-4da9-b12f-03856044f0e6 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:03.376529 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:03.407153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d765dd-257b-4601-bb72-28e6c2c12270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:03.407153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d765dd-257b-4601-bb72-28e6c2c12270 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:03.407153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d765dd-257b-4601-bb72-28e6c2c12270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:03.407153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d765dd-257b-4601-bb72-28e6c2c12270 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:03.407517 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d765dd-257b-4601-bb72-28e6c2c12270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:03.409297 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d765dd-257b-4601-bb72-28e6c2c12270 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:03.422781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1154/2328] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:57:03.424693 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37758a9a-365b-42cc-a7f8-da0007134c07 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:03.435243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37758a9a-365b-42cc-a7f8-da0007134c07 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:03.440419 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fc66f-758c-42b3-8859-33aa0fd6f53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:03.441071 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fc66f-758c-42b3-8859-33aa0fd6f53e admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:03.441352 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fc66f-758c-42b3-8859-33aa0fd6f53e admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:03.441623 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fc66f-758c-42b3-8859-33aa0fd6f53e admin admin] PATH_INFO: `/v3/projects/35c132ce49124f0e8866510cf8f5e890/users/b8b9a024c3b942a6a188818f3e6c055f/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:03.443529 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890fc66f-758c-42b3-8859-33aa0fd6f53e admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c132ce49124f0e8866510cf8f5e890, user_id=b8b9a024c3b942a6a188818f3e6c055f, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:03.446218 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890fc66f-758c-42b3-8859-33aa0fd6f53e admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:03.457864 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1175/2329] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:57:03 2022] PUT /identity/v3/projects/35c132ce49124f0e8866510cf8f5e890/users/b8b9a024c3b942a6a188818f3e6c055f/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:03.468928 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5ef2c-d620-40ab-8f37-53e23009118c None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:03.469158 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5ef2c-d620-40ab-8f37-53e23009118c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:03.469436 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5ef2c-d620-40ab-8f37-53e23009118c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:03.499072 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-10f5ef2c-d620-40ab-8f37-53e23009118c None None] MFA Rules not processed for user `b8b9a024c3b942a6a188818f3e6c055f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:03.500327 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-10f5ef2c-d620-40ab-8f37-53e23009118c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:03.573533 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1155/2330] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:03.584692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fc66f-758c-42b3-8859-33aa0fd6f53e admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:03.586925 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-890fc66f-758c-42b3-8859-33aa0fd6f53e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:03.659721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fc66f-758c-42b3-8859-33aa0fd6f53e admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:03.663774 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc01cb-978d-45b7-8d49-d89ff0c66697 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:03.664388 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc01cb-978d-45b7-8d49-d89ff0c66697 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:03.664669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc01cb-978d-45b7-8d49-d89ff0c66697 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:03.664942 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc01cb-978d-45b7-8d49-d89ff0c66697 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:03.719179 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:03.742166 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc01cb-978d-45b7-8d49-d89ff0c66697 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:03.743665 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc01cb-978d-45b7-8d49-d89ff0c66697 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:03.752801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1176/2331] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:57:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:57:12.522401 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5ef2c-d620-40ab-8f37-53e23009118c None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:12.532006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5ef2c-d620-40ab-8f37-53e23009118c None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:12.535459 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d62f4a-c1af-4f17-8822-d8b847249187 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:12.536130 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d62f4a-c1af-4f17-8822-d8b847249187 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:12.536411 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d62f4a-c1af-4f17-8822-d8b847249187 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:12.536689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d62f4a-c1af-4f17-8822-d8b847249187 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:12.545976 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d62f4a-c1af-4f17-8822-d8b847249187 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:12.547410 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d62f4a-c1af-4f17-8822-d8b847249187 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:12.556880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1156/2332] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:57:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:57:13.139401 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc01cb-978d-45b7-8d49-d89ff0c66697 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:13.140856 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-94fc01cb-978d-45b7-8d49-d89ff0c66697 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:13.196835 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc01cb-978d-45b7-8d49-d89ff0c66697 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:13.200439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512483ee-604b-43e3-9408-e225d7f597f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:13.201042 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512483ee-604b-43e3-9408-e225d7f597f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:13.201320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512483ee-604b-43e3-9408-e225d7f597f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:13.201589 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512483ee-604b-43e3-9408-e225d7f597f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:13.203312 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-512483ee-604b-43e3-9408-e225d7f597f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:13.263752 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512483ee-604b-43e3-9408-e225d7f597f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:13.265217 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512483ee-604b-43e3-9408-e225d7f597f1 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:13.275292 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1177/2333] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:57:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:57:16.531490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d62f4a-c1af-4f17-8822-d8b847249187 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:16.533457 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-75d62f4a-c1af-4f17-8822-d8b847249187 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:16.545160 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d62f4a-c1af-4f17-8822-d8b847249187 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:16.549437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8558b-fe98-4dd7-84c1-19706e943f8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:16.550038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8558b-fe98-4dd7-84c1-19706e943f8b service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:16.550310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8558b-fe98-4dd7-84c1-19706e943f8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:16.550584 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8558b-fe98-4dd7-84c1-19706e943f8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:16.552845 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1be8558b-fe98-4dd7-84c1-19706e943f8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:16.627415 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8558b-fe98-4dd7-84c1-19706e943f8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:16.628880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8558b-fe98-4dd7-84c1-19706e943f8b service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:16.638902 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1157/2334] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:57:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:57:19.368444 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512483ee-604b-43e3-9408-e225d7f597f1 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:19.369836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-512483ee-604b-43e3-9408-e225d7f597f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:19.436903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512483ee-604b-43e3-9408-e225d7f597f1 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:19.441433 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c43a6-1169-4983-a173-e22f19d9c1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:19.442092 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c43a6-1169-4983-a173-e22f19d9c1a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:19.442926 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c43a6-1169-4983-a173-e22f19d9c1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:19.443203 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c43a6-1169-4983-a173-e22f19d9c1a7 admin admin] PATH_INFO: `/v3/users/b4116bba47f74b399a01831a66e1a213` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:19.444541 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c43a6-1169-4983-a173-e22f19d9c1a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4116bba47f74b399a01831a66e1a213)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:19.446538 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c43a6-1169-4983-a173-e22f19d9c1a7 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:19.481576 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:19.495336 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-bd4c43a6-1169-4983-a173-e22f19d9c1a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4116bba47f74b399a01831a66e1a213'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:19.508002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-bd4c43a6-1169-4983-a173-e22f19d9c1a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4116bba47f74b399a01831a66e1a213'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:19.515153 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-bd4c43a6-1169-4983-a173-e22f19d9c1a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4116bba47f74b399a01831a66e1a213'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:19.515817 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-bd4c43a6-1169-4983-a173-e22f19d9c1a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4116bba47f74b399a01831a66e1a213'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:19.523371 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-bd4c43a6-1169-4983-a173-e22f19d9c1a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4116bba47f74b399a01831a66e1a213'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:19.529749 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1178/2335] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:57:19 2022] DELETE /identity/v3/users/b4116bba47f74b399a01831a66e1a213 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:19.534538 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c9c65-e6a0-4721-b161-253c20a4cab8 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:19.535117 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c9c65-e6a0-4721-b161-253c20a4cab8 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:19.535117 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c9c65-e6a0-4721-b161-253c20a4cab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:19.554377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-743c9c65-e6a0-4721-b161-253c20a4cab8 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:19.555599 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-743c9c65-e6a0-4721-b161-253c20a4cab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:19.581322 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:19.614751 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1158/2336] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:19.624545 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c43a6-1169-4983-a173-e22f19d9c1a7 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:19.625941 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-bd4c43a6-1169-4983-a173-e22f19d9c1a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:19.682734 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c43a6-1169-4983-a173-e22f19d9c1a7 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:19.686382 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd26d0-d5fe-479d-8a33-b0ff6bcf00bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:19.686926 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd26d0-d5fe-479d-8a33-b0ff6bcf00bc service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:19.687206 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd26d0-d5fe-479d-8a33-b0ff6bcf00bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:19.687476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd26d0-d5fe-479d-8a33-b0ff6bcf00bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:19.696955 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd26d0-d5fe-479d-8a33-b0ff6bcf00bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:19.698502 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd26d0-d5fe-479d-8a33-b0ff6bcf00bc service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:19.709758 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1179/2337] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:57:19.987282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c9c65-e6a0-4721-b161-253c20a4cab8 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:19.988760 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-743c9c65-e6a0-4721-b161-253c20a4cab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:20.001452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c9c65-e6a0-4721-b161-253c20a4cab8 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:20.006096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0311428-e045-4ba7-bdba-7bde14b18491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:20.006692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0311428-e045-4ba7-bdba-7bde14b18491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:20.006976 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0311428-e045-4ba7-bdba-7bde14b18491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:20.007243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0311428-e045-4ba7-bdba-7bde14b18491 admin admin] PATH_INFO: `/v3/projects/1c5f0bd0da4b4235b22eb4f4e0453034` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:20.008411 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0311428-e045-4ba7-bdba-7bde14b18491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c5f0bd0da4b4235b22eb4f4e0453034)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:20.010569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0311428-e045-4ba7-bdba-7bde14b18491 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:20.056929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e0311428-e045-4ba7-bdba-7bde14b18491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c5f0bd0da4b4235b22eb4f4e0453034', 'request_id': 'req-e0311428-e045-4ba7-bdba-7bde14b18491'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:20.063763 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e0311428-e045-4ba7-bdba-7bde14b18491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c5f0bd0da4b4235b22eb4f4e0453034', 'request_id': 'req-e0311428-e045-4ba7-bdba-7bde14b18491'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:20.069284 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e0311428-e045-4ba7-bdba-7bde14b18491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c5f0bd0da4b4235b22eb4f4e0453034', 'request_id': 'req-e0311428-e045-4ba7-bdba-7bde14b18491'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:20.080225 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e0311428-e045-4ba7-bdba-7bde14b18491 admin admin] The token cache is being invalidate because project 1c5f0bd0da4b4235b22eb4f4e0453034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:57:20.080506 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e0311428-e045-4ba7-bdba-7bde14b18491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:20.081123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-e0311428-e045-4ba7-bdba-7bde14b18491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:20.082572 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1159/2338] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:57:19 2022] DELETE /identity/v3/projects/1c5f0bd0da4b4235b22eb4f4e0453034 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:20.088020 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9fa55-dcf5-43d6-a84f-75894bd762c7 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:20.088371 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9fa55-dcf5-43d6-a84f-75894bd762c7 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:20.088716 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9fa55-dcf5-43d6-a84f-75894bd762c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:20.108242 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-35c9fa55-dcf5-43d6-a84f-75894bd762c7 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:20.109439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-35c9fa55-dcf5-43d6-a84f-75894bd762c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:20.167554 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1180/2339] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:20.176539 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da27183-e996-478a-b41d-745f61703723 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:20.177385 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da27183-e996-478a-b41d-745f61703723 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:20.177682 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da27183-e996-478a-b41d-745f61703723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:20.196859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-2da27183-e996-478a-b41d-745f61703723 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:20.198059 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-2da27183-e996-478a-b41d-745f61703723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:20.205813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1160/2340] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:20.209980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9fa55-dcf5-43d6-a84f-75894bd762c7 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:20.219695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9fa55-dcf5-43d6-a84f-75894bd762c7 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:20.224810 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784af1c7-c62b-4d6e-b501-73d5d58758d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:20.225408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784af1c7-c62b-4d6e-b501-73d5d58758d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:20.225686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784af1c7-c62b-4d6e-b501-73d5d58758d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:20.225961 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784af1c7-c62b-4d6e-b501-73d5d58758d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:20.226598 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784af1c7-c62b-4d6e-b501-73d5d58758d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:57:20.226902 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784af1c7-c62b-4d6e-b501-73d5d58758d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:20.228537 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784af1c7-c62b-4d6e-b501-73d5d58758d5 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:20.240479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1181/2341] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:57:20.244431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da27183-e996-478a-b41d-745f61703723 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:20.253417 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da27183-e996-478a-b41d-745f61703723 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:20.257982 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dd9f6-6cbc-4955-aa21-47571ec9ec09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:20.258568 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dd9f6-6cbc-4955-aa21-47571ec9ec09 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:20.258846 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dd9f6-6cbc-4955-aa21-47571ec9ec09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:20.259118 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dd9f6-6cbc-4955-aa21-47571ec9ec09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:20.260058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849dd9f6-6cbc-4955-aa21-47571ec9ec09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:20.261933 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849dd9f6-6cbc-4955-aa21-47571ec9ec09 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:20.288845 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1161/2342] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:57:20 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:57:20.292955 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784af1c7-c62b-4d6e-b501-73d5d58758d5 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:20.302677 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784af1c7-c62b-4d6e-b501-73d5d58758d5 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:20.335772 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:20.360831 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ef605-3beb-4410-9333-d270fb2cb785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:20.361442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ef605-3beb-4410-9333-d270fb2cb785 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:20.361721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ef605-3beb-4410-9333-d270fb2cb785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:20.361994 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ef605-3beb-4410-9333-d270fb2cb785 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:20.362647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ef605-3beb-4410-9333-d270fb2cb785 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:20.364609 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ef605-3beb-4410-9333-d270fb2cb785 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:20.389913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1182/2343] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:57:20 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:57:20.391160 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dd9f6-6cbc-4955-aa21-47571ec9ec09 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:20.402049 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dd9f6-6cbc-4955-aa21-47571ec9ec09 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:20.406551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf383a-537c-448c-8391-5ee0daddf786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:20.407152 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf383a-537c-448c-8391-5ee0daddf786 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:20.407449 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf383a-537c-448c-8391-5ee0daddf786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:20.407746 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf383a-537c-448c-8391-5ee0daddf786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:20.408376 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf383a-537c-448c-8391-5ee0daddf786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:20.409950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf383a-537c-448c-8391-5ee0daddf786 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:20.420686 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1162/2344] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:57:20.424504 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ef605-3beb-4410-9333-d270fb2cb785 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:20.435038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ef605-3beb-4410-9333-d270fb2cb785 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:20.440109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811a809-9e7c-4c37-a726-f34f45bd6846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:20.440719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811a809-9e7c-4c37-a726-f34f45bd6846 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:20.440999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811a809-9e7c-4c37-a726-f34f45bd6846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:20.441270 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811a809-9e7c-4c37-a726-f34f45bd6846 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:20.441912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811a809-9e7c-4c37-a726-f34f45bd6846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:20.443548 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811a809-9e7c-4c37-a726-f34f45bd6846 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:20.454483 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1183/2345] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:57:20.459144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf383a-537c-448c-8391-5ee0daddf786 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:20.469005 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf383a-537c-448c-8391-5ee0daddf786 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:20.474325 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a5594-41fa-42cf-9dd6-80e169a57ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:20.474967 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a5594-41fa-42cf-9dd6-80e169a57ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:20.475248 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a5594-41fa-42cf-9dd6-80e169a57ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:20.475519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a5594-41fa-42cf-9dd6-80e169a57ee1 admin admin] PATH_INFO: `/v3/projects/9eeb7760618344fe939ef9b6ea66c922/users/19f677fff2bd4175806faded8310c3a6/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:20.490027 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a5594-41fa-42cf-9dd6-80e169a57ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eeb7760618344fe939ef9b6ea66c922, user_id=19f677fff2bd4175806faded8310c3a6, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:20.492776 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a5594-41fa-42cf-9dd6-80e169a57ee1 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:20.507335 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1163/2346] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:57:20 2022] PUT /identity/v3/projects/9eeb7760618344fe939ef9b6ea66c922/users/19f677fff2bd4175806faded8310c3a6/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:20.518375 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd27e7-b8ab-4c4e-975f-a49ab21bbd0b None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:20.518742 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd27e7-b8ab-4c4e-975f-a49ab21bbd0b None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:20.519087 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd27e7-b8ab-4c4e-975f-a49ab21bbd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:20.548639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-77fd27e7-b8ab-4c4e-975f-a49ab21bbd0b None None] MFA Rules not processed for user `19f677fff2bd4175806faded8310c3a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:20.549952 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-77fd27e7-b8ab-4c4e-975f-a49ab21bbd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:20.624291 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1184/2347] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:20.628967 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a5594-41fa-42cf-9dd6-80e169a57ee1 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:20.679398 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:20.691642 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a5594-41fa-42cf-9dd6-80e169a57ee1 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:20.697576 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212ef2d-b4f2-45c7-8d40-e122ac8d032c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:20.698202 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212ef2d-b4f2-45c7-8d40-e122ac8d032c admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:20.698481 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212ef2d-b4f2-45c7-8d40-e122ac8d032c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:20.698745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212ef2d-b4f2-45c7-8d40-e122ac8d032c admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:20.699444 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d212ef2d-b4f2-45c7-8d40-e122ac8d032c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:20.701475 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d212ef2d-b4f2-45c7-8d40-e122ac8d032c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:20.728549 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1164/2348] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:57:20 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:57:20.732661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd27e7-b8ab-4c4e-975f-a49ab21bbd0b None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:20.743310 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd27e7-b8ab-4c4e-975f-a49ab21bbd0b None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:20.796639 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75937f1c-d2aa-4250-a88e-421c8a7ea691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:20.797233 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75937f1c-d2aa-4250-a88e-421c8a7ea691 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:20.797531 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75937f1c-d2aa-4250-a88e-421c8a7ea691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:20.797800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75937f1c-d2aa-4250-a88e-421c8a7ea691 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:20.798405 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75937f1c-d2aa-4250-a88e-421c8a7ea691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:20.800441 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75937f1c-d2aa-4250-a88e-421c8a7ea691 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:20.821097 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1185/2349] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:57:20 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:57:20.825255 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212ef2d-b4f2-45c7-8d40-e122ac8d032c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:20.836294 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212ef2d-b4f2-45c7-8d40-e122ac8d032c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:20.840842 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb78476-58eb-42ad-b2eb-28cb4d5de0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:20.841437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb78476-58eb-42ad-b2eb-28cb4d5de0ac admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:20.841718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb78476-58eb-42ad-b2eb-28cb4d5de0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:20.841992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb78476-58eb-42ad-b2eb-28cb4d5de0ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:20.842633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb78476-58eb-42ad-b2eb-28cb4d5de0ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:20.844254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb78476-58eb-42ad-b2eb-28cb4d5de0ac admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:20.854832 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1165/2350] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:57:20.858683 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75937f1c-d2aa-4250-a88e-421c8a7ea691 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:20.868493 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75937f1c-d2aa-4250-a88e-421c8a7ea691 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:20.873114 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85584fe2-2c5e-4ba9-9ae6-8fe6996aa12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:20.873698 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85584fe2-2c5e-4ba9-9ae6-8fe6996aa12c admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:20.873973 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85584fe2-2c5e-4ba9-9ae6-8fe6996aa12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:20.874242 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85584fe2-2c5e-4ba9-9ae6-8fe6996aa12c admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:20.874866 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85584fe2-2c5e-4ba9-9ae6-8fe6996aa12c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:20.876491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85584fe2-2c5e-4ba9-9ae6-8fe6996aa12c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:20.886833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1186/2351] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:57:20.890715 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb78476-58eb-42ad-b2eb-28cb4d5de0ac admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:20.901167 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb78476-58eb-42ad-b2eb-28cb4d5de0ac admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:20.905733 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edf29b-c0a0-4f84-8dbc-11f7c74b6aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:20.906478 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edf29b-c0a0-4f84-8dbc-11f7c74b6aa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:20.906764 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edf29b-c0a0-4f84-8dbc-11f7c74b6aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:20.907040 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edf29b-c0a0-4f84-8dbc-11f7c74b6aa9 admin admin] PATH_INFO: `/v3/projects/c2b12cf6de184910bcc5dda7d212938b/users/93c2a1953f2c4acead7559d8b9be66bd/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:20.921529 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57edf29b-c0a0-4f84-8dbc-11f7c74b6aa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2b12cf6de184910bcc5dda7d212938b, user_id=93c2a1953f2c4acead7559d8b9be66bd, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:20.924199 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57edf29b-c0a0-4f84-8dbc-11f7c74b6aa9 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:20.938889 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1166/2352] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:57:20 2022] PUT /identity/v3/projects/c2b12cf6de184910bcc5dda7d212938b/users/93c2a1953f2c4acead7559d8b9be66bd/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:20.949875 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8de961-90fa-4f9e-b83d-1da924e22418 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:20.950162 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8de961-90fa-4f9e-b83d-1da924e22418 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:20.950431 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8de961-90fa-4f9e-b83d-1da924e22418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:20.978161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-cd8de961-90fa-4f9e-b83d-1da924e22418 None None] MFA Rules not processed for user `93c2a1953f2c4acead7559d8b9be66bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:20.979346 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-cd8de961-90fa-4f9e-b83d-1da924e22418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:20.990706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:21.043562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1187/2353] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:21.048097 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edf29b-c0a0-4f84-8dbc-11f7c74b6aa9 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:21.108479 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edf29b-c0a0-4f84-8dbc-11f7c74b6aa9 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:21.113027 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398f051-2003-449d-851a-d15509c8908f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:21.113618 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398f051-2003-449d-851a-d15509c8908f admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:21.113899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398f051-2003-449d-851a-d15509c8908f admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:21.114660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398f051-2003-449d-851a-d15509c8908f admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:21.115280 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398f051-2003-449d-851a-d15509c8908f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:21.117170 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398f051-2003-449d-851a-d15509c8908f admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:21.142788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1167/2354] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:57:21 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:57:21.146852 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8de961-90fa-4f9e-b83d-1da924e22418 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:21.157956 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8de961-90fa-4f9e-b83d-1da924e22418 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:21.212543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94033a54-0dbe-4b09-bb4a-5f69ce52a206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:21.213171 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94033a54-0dbe-4b09-bb4a-5f69ce52a206 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:21.213450 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94033a54-0dbe-4b09-bb4a-5f69ce52a206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:21.213721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94033a54-0dbe-4b09-bb4a-5f69ce52a206 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:21.214348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94033a54-0dbe-4b09-bb4a-5f69ce52a206 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:21.216331 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94033a54-0dbe-4b09-bb4a-5f69ce52a206 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:21.237033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1188/2355] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:57:21 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:57:21.241466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398f051-2003-449d-851a-d15509c8908f admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:21.251507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398f051-2003-449d-851a-d15509c8908f admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:21.256730 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de1481-4c52-42a9-ab29-d033d3cf6cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:21.257320 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de1481-4c52-42a9-ab29-d033d3cf6cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:21.257602 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de1481-4c52-42a9-ab29-d033d3cf6cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:21.257877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de1481-4c52-42a9-ab29-d033d3cf6cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:21.258514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de1481-4c52-42a9-ab29-d033d3cf6cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:21.260401 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de1481-4c52-42a9-ab29-d033d3cf6cc2 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:21.271024 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1168/2356] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:57:21.274824 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94033a54-0dbe-4b09-bb4a-5f69ce52a206 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:21.284601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94033a54-0dbe-4b09-bb4a-5f69ce52a206 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:21.289150 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3090742-afe2-4e5d-b578-2c3fab128384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:21.289789 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3090742-afe2-4e5d-b578-2c3fab128384 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:21.290064 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3090742-afe2-4e5d-b578-2c3fab128384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:21.290334 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3090742-afe2-4e5d-b578-2c3fab128384 admin admin] PATH_INFO: `/v3/projects/e2e5529f2dc64e1a823d2788057aad78/users/eba19f4c6a164a608de737a1a901c011/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:21.305230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3090742-afe2-4e5d-b578-2c3fab128384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2e5529f2dc64e1a823d2788057aad78, user_id=eba19f4c6a164a608de737a1a901c011, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:21.307862 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3090742-afe2-4e5d-b578-2c3fab128384 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:21.320980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1189/2357] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:57:21 2022] PUT /identity/v3/projects/e2e5529f2dc64e1a823d2788057aad78/users/eba19f4c6a164a608de737a1a901c011/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:21.324969 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de1481-4c52-42a9-ab29-d033d3cf6cc2 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:21.334891 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de1481-4c52-42a9-ab29-d033d3cf6cc2 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:21.390292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ea39c-8fc6-4aa7-897f-718623eb0246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:21.390891 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ea39c-8fc6-4aa7-897f-718623eb0246 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:21.391174 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ea39c-8fc6-4aa7-897f-718623eb0246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:21.391449 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ea39c-8fc6-4aa7-897f-718623eb0246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:21.392109 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ea39c-8fc6-4aa7-897f-718623eb0246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:21.393689 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ea39c-8fc6-4aa7-897f-718623eb0246 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:21.403086 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1169/2358] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:57:21.407123 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3090742-afe2-4e5d-b578-2c3fab128384 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:21.419195 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3090742-afe2-4e5d-b578-2c3fab128384 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:21.424052 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc368d-5443-4832-a567-a924f17616e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:21.424701 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc368d-5443-4832-a567-a924f17616e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:21.424979 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc368d-5443-4832-a567-a924f17616e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:21.425247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc368d-5443-4832-a567-a924f17616e2 admin admin] PATH_INFO: `/v3/projects/e2e5529f2dc64e1a823d2788057aad78/users/eba19f4c6a164a608de737a1a901c011/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:21.427794 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc368d-5443-4832-a567-a924f17616e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2e5529f2dc64e1a823d2788057aad78, user_id=eba19f4c6a164a608de737a1a901c011, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:21.430432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc368d-5443-4832-a567-a924f17616e2 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:21.440544 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1190/2359] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:57:21 2022] PUT /identity/v3/projects/e2e5529f2dc64e1a823d2788057aad78/users/eba19f4c6a164a608de737a1a901c011/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:21.451444 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178bdc8-657f-4f2b-a267-ff8a1a4539bb None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:21.451791 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178bdc8-657f-4f2b-a267-ff8a1a4539bb None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:21.452070 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178bdc8-657f-4f2b-a267-ff8a1a4539bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:21.471749 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:21.480076 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-a178bdc8-657f-4f2b-a267-ff8a1a4539bb None None] MFA Rules not processed for user `eba19f4c6a164a608de737a1a901c011`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:21.481648 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a178bdc8-657f-4f2b-a267-ff8a1a4539bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:21.554804 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1170/2360] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:21.565938 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc368d-5443-4832-a567-a924f17616e2 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:21.567756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d2fc368d-5443-4832-a567-a924f17616e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:21.635132 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc368d-5443-4832-a567-a924f17616e2 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:21.639871 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88af0f0-376d-4a33-9273-96f41d09e9a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:21.640481 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88af0f0-376d-4a33-9273-96f41d09e9a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:21.640830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88af0f0-376d-4a33-9273-96f41d09e9a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:21.641055 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88af0f0-376d-4a33-9273-96f41d09e9a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:21.662752 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:21.704328 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88af0f0-376d-4a33-9273-96f41d09e9a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:21.705798 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88af0f0-376d-4a33-9273-96f41d09e9a5 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:21.716197 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1191/2361] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:57:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:57:26.389703 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178bdc8-657f-4f2b-a267-ff8a1a4539bb None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:26.391185 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a178bdc8-657f-4f2b-a267-ff8a1a4539bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:26.403787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178bdc8-657f-4f2b-a267-ff8a1a4539bb None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:26.407226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306efbc-0f1f-42a3-a858-b0485772a894 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:26.407856 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306efbc-0f1f-42a3-a858-b0485772a894 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:26.408129 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306efbc-0f1f-42a3-a858-b0485772a894 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:26.408394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306efbc-0f1f-42a3-a858-b0485772a894 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:26.463516 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1306efbc-0f1f-42a3-a858-b0485772a894 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:26.464992 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1306efbc-0f1f-42a3-a858-b0485772a894 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:26.474336 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1171/2362] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:57:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:57:26.992551 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88af0f0-376d-4a33-9273-96f41d09e9a5 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:27.002745 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88af0f0-376d-4a33-9273-96f41d09e9a5 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:27.006975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad3779-ee78-42a9-995b-a4b80e83d8c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:27.007749 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad3779-ee78-42a9-995b-a4b80e83d8c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:27.008096 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad3779-ee78-42a9-995b-a4b80e83d8c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:27.008433 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad3779-ee78-42a9-995b-a4b80e83d8c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:27.018184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad3779-ee78-42a9-995b-a4b80e83d8c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:27.019748 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad3779-ee78-42a9-995b-a4b80e83d8c9 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:27.030330 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1192/2363] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:57:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:57:35.110366 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306efbc-0f1f-42a3-a858-b0485772a894 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:35.171016 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306efbc-0f1f-42a3-a858-b0485772a894 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:35.175403 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78777fd-1f76-47c1-af2c-1570e01b38ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:35.176047 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78777fd-1f76-47c1-af2c-1570e01b38ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:35.176323 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78777fd-1f76-47c1-af2c-1570e01b38ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:35.176591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78777fd-1f76-47c1-af2c-1570e01b38ee admin admin] PATH_INFO: `/v3/users/19f677fff2bd4175806faded8310c3a6` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:35.178228 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78777fd-1f76-47c1-af2c-1570e01b38ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=19f677fff2bd4175806faded8310c3a6)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:35.180164 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78777fd-1f76-47c1-af2c-1570e01b38ee admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:35.215285 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:35.224419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-d78777fd-1f76-47c1-af2c-1570e01b38ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19f677fff2bd4175806faded8310c3a6'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:35.225062 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-d78777fd-1f76-47c1-af2c-1570e01b38ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19f677fff2bd4175806faded8310c3a6'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:35.236546 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-d78777fd-1f76-47c1-af2c-1570e01b38ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19f677fff2bd4175806faded8310c3a6'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:35.244015 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-d78777fd-1f76-47c1-af2c-1570e01b38ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19f677fff2bd4175806faded8310c3a6'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:35.244609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-d78777fd-1f76-47c1-af2c-1570e01b38ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19f677fff2bd4175806faded8310c3a6'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:35.256621 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1172/2364] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:57:35 2022] DELETE /identity/v3/users/19f677fff2bd4175806faded8310c3a6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:35.261225 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef3632-c8ce-4b41-80fb-c9dccf2d5de3 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:35.261675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef3632-c8ce-4b41-80fb-c9dccf2d5de3 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:35.261761 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef3632-c8ce-4b41-80fb-c9dccf2d5de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:35.289767 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-8eef3632-c8ce-4b41-80fb-c9dccf2d5de3 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:35.291029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8eef3632-c8ce-4b41-80fb-c9dccf2d5de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:35.347655 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1193/2365] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:35.357873 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78777fd-1f76-47c1-af2c-1570e01b38ee admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:35.359256 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d78777fd-1f76-47c1-af2c-1570e01b38ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:35.415504 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78777fd-1f76-47c1-af2c-1570e01b38ee admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:35.419095 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aab8b0-c2a4-46fb-bcc4-f99e44c0a088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:35.419726 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aab8b0-c2a4-46fb-bcc4-f99e44c0a088 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:35.420005 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aab8b0-c2a4-46fb-bcc4-f99e44c0a088 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:35.420274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aab8b0-c2a4-46fb-bcc4-f99e44c0a088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:35.429039 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aab8b0-c2a4-46fb-bcc4-f99e44c0a088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:35.430448 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aab8b0-c2a4-46fb-bcc4-f99e44c0a088 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:35.439722 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1173/2366] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:57:35.699088 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef3632-c8ce-4b41-80fb-c9dccf2d5de3 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:35.700641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8eef3632-c8ce-4b41-80fb-c9dccf2d5de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:35.714091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef3632-c8ce-4b41-80fb-c9dccf2d5de3 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:35.718889 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:35.719538 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:35.719826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:35.720094 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f admin admin] PATH_INFO: `/v3/projects/9eeb7760618344fe939ef9b6ea66c922` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:35.721734 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eeb7760618344fe939ef9b6ea66c922)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:35.723941 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:35.777800 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eeb7760618344fe939ef9b6ea66c922', 'request_id': 'req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:35.784167 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eeb7760618344fe939ef9b6ea66c922', 'request_id': 'req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:35.792163 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eeb7760618344fe939ef9b6ea66c922', 'request_id': 'req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:35.802903 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f admin admin] The token cache is being invalidate because project 9eeb7760618344fe939ef9b6ea66c922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:57:35.803210 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:35.803873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:35.805376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1194/2367] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:57:35 2022] DELETE /identity/v3/projects/9eeb7760618344fe939ef9b6ea66c922 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:35.809287 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aab8b0-c2a4-46fb-bcc4-f99e44c0a088 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:35.819138 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aab8b0-c2a4-46fb-bcc4-f99e44c0a088 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:35.874405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93171221-ad29-4b42-bded-553924cbf174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:35.875082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93171221-ad29-4b42-bded-553924cbf174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:35.875302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93171221-ad29-4b42-bded-553924cbf174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:35.875591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93171221-ad29-4b42-bded-553924cbf174 admin admin] PATH_INFO: `/v3/users/93c2a1953f2c4acead7559d8b9be66bd` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:35.876731 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93171221-ad29-4b42-bded-553924cbf174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c2a1953f2c4acead7559d8b9be66bd)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:35.878653 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93171221-ad29-4b42-bded-553924cbf174 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:35.924688 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-93171221-ad29-4b42-bded-553924cbf174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c2a1953f2c4acead7559d8b9be66bd'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:35.925326 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-93171221-ad29-4b42-bded-553924cbf174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c2a1953f2c4acead7559d8b9be66bd'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:35.932763 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:35.937789 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-93171221-ad29-4b42-bded-553924cbf174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c2a1953f2c4acead7559d8b9be66bd'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:35.946177 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-93171221-ad29-4b42-bded-553924cbf174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c2a1953f2c4acead7559d8b9be66bd'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:35.946816 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-93171221-ad29-4b42-bded-553924cbf174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c2a1953f2c4acead7559d8b9be66bd'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:35.959441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1174/2368] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:57:35 2022] DELETE /identity/v3/users/93c2a1953f2c4acead7559d8b9be66bd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:36.226926 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:36.228372 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:36.241397 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd766ae1-8cc6-4f2b-b603-cea8253fb60f admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:36.262878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:36.301144 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:36.301802 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:36.302086 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:36.302358 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7 admin admin] PATH_INFO: `/v3/projects/c2b12cf6de184910bcc5dda7d212938b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:36.303484 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2b12cf6de184910bcc5dda7d212938b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:36.305416 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:36.350188 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2b12cf6de184910bcc5dda7d212938b', 'request_id': 'req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:36.356339 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2b12cf6de184910bcc5dda7d212938b', 'request_id': 'req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:36.363481 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2b12cf6de184910bcc5dda7d212938b', 'request_id': 'req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:36.373521 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7 admin admin] The token cache is being invalidate because project c2b12cf6de184910bcc5dda7d212938b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:57:36.373803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:36.374420 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:36.375908 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1195/2369] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:57:36 2022] DELETE /identity/v3/projects/c2b12cf6de184910bcc5dda7d212938b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:36.380448 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93171221-ad29-4b42-bded-553924cbf174 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:36.381837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-93171221-ad29-4b42-bded-553924cbf174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:36.394665 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93171221-ad29-4b42-bded-553924cbf174 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:36.449502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a95870-ccc5-4c64-8a46-52ec375b7f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:36.450119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a95870-ccc5-4c64-8a46-52ec375b7f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:36.450396 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a95870-ccc5-4c64-8a46-52ec375b7f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:36.450666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a95870-ccc5-4c64-8a46-52ec375b7f47 admin admin] PATH_INFO: `/v3/users/eba19f4c6a164a608de737a1a901c011` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:36.451784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a95870-ccc5-4c64-8a46-52ec375b7f47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eba19f4c6a164a608de737a1a901c011)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:36.453684 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a95870-ccc5-4c64-8a46-52ec375b7f47 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:36.497688 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b6a95870-ccc5-4c64-8a46-52ec375b7f47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eba19f4c6a164a608de737a1a901c011'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:36.498357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b6a95870-ccc5-4c64-8a46-52ec375b7f47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eba19f4c6a164a608de737a1a901c011'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:36.510759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b6a95870-ccc5-4c64-8a46-52ec375b7f47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eba19f4c6a164a608de737a1a901c011'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:36.517936 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b6a95870-ccc5-4c64-8a46-52ec375b7f47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eba19f4c6a164a608de737a1a901c011'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:36.518548 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b6a95870-ccc5-4c64-8a46-52ec375b7f47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eba19f4c6a164a608de737a1a901c011'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:36.530924 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1175/2370] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:57:36 2022] DELETE /identity/v3/users/eba19f4c6a164a608de737a1a901c011 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:36.802721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:36.804343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:36.817826 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efc3ed-4acc-41c0-ae52-d7410d7ef7c7 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:36.876214 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1350275-2653-426e-8c79-50e1003c35bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:36.876847 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1350275-2653-426e-8c79-50e1003c35bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:36.877127 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1350275-2653-426e-8c79-50e1003c35bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:36.877400 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1350275-2653-426e-8c79-50e1003c35bd admin admin] PATH_INFO: `/v3/projects/e2e5529f2dc64e1a823d2788057aad78` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:36.878518 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1350275-2653-426e-8c79-50e1003c35bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2e5529f2dc64e1a823d2788057aad78)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:36.880513 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1350275-2653-426e-8c79-50e1003c35bd admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:36.930432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e1350275-2653-426e-8c79-50e1003c35bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2e5529f2dc64e1a823d2788057aad78', 'request_id': 'req-e1350275-2653-426e-8c79-50e1003c35bd'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:36.936590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e1350275-2653-426e-8c79-50e1003c35bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2e5529f2dc64e1a823d2788057aad78', 'request_id': 'req-e1350275-2653-426e-8c79-50e1003c35bd'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:36.939082 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:36.944090 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e1350275-2653-426e-8c79-50e1003c35bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2e5529f2dc64e1a823d2788057aad78', 'request_id': 'req-e1350275-2653-426e-8c79-50e1003c35bd'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:36.955849 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e1350275-2653-426e-8c79-50e1003c35bd admin admin] The token cache is being invalidate because project e2e5529f2dc64e1a823d2788057aad78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:57:36.956132 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e1350275-2653-426e-8c79-50e1003c35bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:36.956833 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e1350275-2653-426e-8c79-50e1003c35bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:36.958906 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1196/2371] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:57:36 2022] DELETE /identity/v3/projects/e2e5529f2dc64e1a823d2788057aad78 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:36.963872 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fd1aa-df62-451a-9f52-794e505611fe None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:36.964164 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fd1aa-df62-451a-9f52-794e505611fe None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:36.964440 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fd1aa-df62-451a-9f52-794e505611fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:37.011809 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-635fd1aa-df62-451a-9f52-794e505611fe None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:37.012935 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-635fd1aa-df62-451a-9f52-794e505611fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:37.039190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:37.072844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1176/2372] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:37.080653 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a68bb-9756-480f-bf10-7f55829e5888 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:37.080938 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a68bb-9756-480f-bf10-7f55829e5888 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:37.081212 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a68bb-9756-480f-bf10-7f55829e5888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:37.101428 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-c57a68bb-9756-480f-bf10-7f55829e5888 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:37.102613 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-c57a68bb-9756-480f-bf10-7f55829e5888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:37.110375 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1197/2373] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:37.114571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635fd1aa-df62-451a-9f52-794e505611fe None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:37.124147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635fd1aa-df62-451a-9f52-794e505611fe None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:37.128772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a1af3-d05b-495f-a2e1-a9b5a156e58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:37.129363 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a1af3-d05b-495f-a2e1-a9b5a156e58d admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:37.129651 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a1af3-d05b-495f-a2e1-a9b5a156e58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:37.130414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a1af3-d05b-495f-a2e1-a9b5a156e58d admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:37.131054 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a1af3-d05b-495f-a2e1-a9b5a156e58d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:57:37.131358 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a1af3-d05b-495f-a2e1-a9b5a156e58d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:37.132975 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a1af3-d05b-495f-a2e1-a9b5a156e58d admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:37.144728 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1177/2374] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:57:37.149324 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a68bb-9756-480f-bf10-7f55829e5888 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:37.158433 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a68bb-9756-480f-bf10-7f55829e5888 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:37.163004 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56954776-1df5-4f38-aa80-70ff77ad11d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:37.163615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56954776-1df5-4f38-aa80-70ff77ad11d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:37.163891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56954776-1df5-4f38-aa80-70ff77ad11d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:37.164159 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56954776-1df5-4f38-aa80-70ff77ad11d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:37.164789 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56954776-1df5-4f38-aa80-70ff77ad11d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:37.166697 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56954776-1df5-4f38-aa80-70ff77ad11d9 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:37.202230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1198/2375] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:57:37 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:57:37.206431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a1af3-d05b-495f-a2e1-a9b5a156e58d admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:37.216674 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a1af3-d05b-495f-a2e1-a9b5a156e58d admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:37.271766 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc92ab-4d54-45e6-b4d2-b37c717fcbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:37.272377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc92ab-4d54-45e6-b4d2-b37c717fcbf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:37.272659 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc92ab-4d54-45e6-b4d2-b37c717fcbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:37.272931 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc92ab-4d54-45e6-b4d2-b37c717fcbf5 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:37.273560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc92ab-4d54-45e6-b4d2-b37c717fcbf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:37.275461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc92ab-4d54-45e6-b4d2-b37c717fcbf5 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:37.296050 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1178/2376] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:57:37 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:57:37.300263 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56954776-1df5-4f38-aa80-70ff77ad11d9 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:37.310587 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56954776-1df5-4f38-aa80-70ff77ad11d9 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:37.315151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df7149-7d8a-4610-ad28-a5b47ace4501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:37.315778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df7149-7d8a-4610-ad28-a5b47ace4501 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:37.316059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df7149-7d8a-4610-ad28-a5b47ace4501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:37.316332 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df7149-7d8a-4610-ad28-a5b47ace4501 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:37.316959 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df7149-7d8a-4610-ad28-a5b47ace4501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:37.319072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df7149-7d8a-4610-ad28-a5b47ace4501 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:37.328717 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1199/2377] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:57:37.332368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc92ab-4d54-45e6-b4d2-b37c717fcbf5 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:37.342608 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc92ab-4d54-45e6-b4d2-b37c717fcbf5 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:37.347092 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00717c6-35d8-4e17-bc98-a5c294549d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:37.347708 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00717c6-35d8-4e17-bc98-a5c294549d6e admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:37.347984 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00717c6-35d8-4e17-bc98-a5c294549d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:37.348254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00717c6-35d8-4e17-bc98-a5c294549d6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:37.348877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00717c6-35d8-4e17-bc98-a5c294549d6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:37.350605 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00717c6-35d8-4e17-bc98-a5c294549d6e admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:37.362143 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1179/2378] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:57:37.366106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df7149-7d8a-4610-ad28-a5b47ace4501 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:37.375651 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df7149-7d8a-4610-ad28-a5b47ace4501 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:37.380648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dfbb2-f23a-4d03-ac25-4f5ef23e7db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:37.381282 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dfbb2-f23a-4d03-ac25-4f5ef23e7db9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:37.381556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dfbb2-f23a-4d03-ac25-4f5ef23e7db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:37.381823 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dfbb2-f23a-4d03-ac25-4f5ef23e7db9 admin admin] PATH_INFO: `/v3/projects/bdafb20ebf674ea6a7f82876c7724341/users/a665b883f5ab4c408329c358da8972cf/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:37.397220 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dfbb2-f23a-4d03-ac25-4f5ef23e7db9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdafb20ebf674ea6a7f82876c7724341, user_id=a665b883f5ab4c408329c358da8972cf, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:37.399931 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dfbb2-f23a-4d03-ac25-4f5ef23e7db9 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:37.412387 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1200/2379] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:57:37 2022] PUT /identity/v3/projects/bdafb20ebf674ea6a7f82876c7724341/users/a665b883f5ab4c408329c358da8972cf/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:57:37.417346 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d0279-6f64-4785-9666-724a9404588b None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:37.417825 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d0279-6f64-4785-9666-724a9404588b None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:37.417930 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d0279-6f64-4785-9666-724a9404588b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:37.436294 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-d41d0279-6f64-4785-9666-724a9404588b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:37.437648 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d41d0279-6f64-4785-9666-724a9404588b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:37.497899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1180/2380] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:37.508039 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dfbb2-f23a-4d03-ac25-4f5ef23e7db9 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:37.509998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-287dfbb2-f23a-4d03-ac25-4f5ef23e7db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:37.563987 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dfbb2-f23a-4d03-ac25-4f5ef23e7db9 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:37.568287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf474e-b967-4240-b085-18ce1469954e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:37.568863 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf474e-b967-4240-b085-18ce1469954e service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:37.569139 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf474e-b967-4240-b085-18ce1469954e service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:37.569411 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf474e-b967-4240-b085-18ce1469954e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:37.578336 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf474e-b967-4240-b085-18ce1469954e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:37.579788 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf474e-b967-4240-b085-18ce1469954e service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:37.588955 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1201/2381] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:57:38.225565 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec88be2-e4f3-456a-9816-18327de8adc8 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:38.225843 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec88be2-e4f3-456a-9816-18327de8adc8 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:38.226116 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec88be2-e4f3-456a-9816-18327de8adc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:38.253968 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-0ec88be2-e4f3-456a-9816-18327de8adc8 None None] MFA Rules not processed for user `a665b883f5ab4c408329c358da8972cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:38.255211 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-0ec88be2-e4f3-456a-9816-18327de8adc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:38.281108 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:38.320082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1181/2382] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:57:38.330408 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf474e-b967-4240-b085-18ce1469954e service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:38.331861 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e1cf474e-b967-4240-b085-18ce1469954e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:38.388567 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf474e-b967-4240-b085-18ce1469954e service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:38.391976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77b52e-04ed-4d60-81ad-019ae2d996be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:38.392579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77b52e-04ed-4d60-81ad-019ae2d996be service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:38.392855 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77b52e-04ed-4d60-81ad-019ae2d996be service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:38.393120 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77b52e-04ed-4d60-81ad-019ae2d996be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:38.401898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77b52e-04ed-4d60-81ad-019ae2d996be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:38.403328 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77b52e-04ed-4d60-81ad-019ae2d996be service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:38.412637 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1202/2383] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:57:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:57:40.811969 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffeecd-a7ba-4c74-9a72-eb9a3ab7090b None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:40.812477 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffeecd-a7ba-4c74-9a72-eb9a3ab7090b None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:40.812592 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffeecd-a7ba-4c74-9a72-eb9a3ab7090b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:40.831820 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-89ffeecd-a7ba-4c74-9a72-eb9a3ab7090b None None] MFA Rules not processed for user `a665b883f5ab4c408329c358da8972cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:40.833018 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-89ffeecd-a7ba-4c74-9a72-eb9a3ab7090b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:40.839873 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1182/2384] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:43.754156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77b52e-04ed-4d60-81ad-019ae2d996be service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:43.756311 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2c77b52e-04ed-4d60-81ad-019ae2d996be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:43.770347 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77b52e-04ed-4d60-81ad-019ae2d996be service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:43.774777 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95c777-ad25-4946-9078-b7e3529339d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:43.775393 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95c777-ad25-4946-9078-b7e3529339d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:43.775708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95c777-ad25-4946-9078-b7e3529339d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:43.775978 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95c777-ad25-4946-9078-b7e3529339d4 admin admin] PATH_INFO: `/v3/users/9e4755af6a1440828d56173bd17a9803` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:43.777097 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95c777-ad25-4946-9078-b7e3529339d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e4755af6a1440828d56173bd17a9803)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:43.779007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95c777-ad25-4946-9078-b7e3529339d4 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:43.791538 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:43.826295 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0f95c777-ad25-4946-9078-b7e3529339d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e4755af6a1440828d56173bd17a9803'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:43.838733 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0f95c777-ad25-4946-9078-b7e3529339d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e4755af6a1440828d56173bd17a9803'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:43.846242 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0f95c777-ad25-4946-9078-b7e3529339d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e4755af6a1440828d56173bd17a9803'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:43.846905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0f95c777-ad25-4946-9078-b7e3529339d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e4755af6a1440828d56173bd17a9803'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:43.854255 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0f95c777-ad25-4946-9078-b7e3529339d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e4755af6a1440828d56173bd17a9803'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:43.860596 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1203/2385] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:57:43 2022] DELETE /identity/v3/users/9e4755af6a1440828d56173bd17a9803 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:43.865838 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e870f3-a098-473c-aa5e-11a40444eb3c None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:43.866312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e870f3-a098-473c-aa5e-11a40444eb3c None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:43.867086 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e870f3-a098-473c-aa5e-11a40444eb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:43.887064 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-a1e870f3-a098-473c-aa5e-11a40444eb3c None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:43.888592 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a1e870f3-a098-473c-aa5e-11a40444eb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:43.949564 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1183/2386] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:57:43.959925 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95c777-ad25-4946-9078-b7e3529339d4 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:43.961507 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0f95c777-ad25-4946-9078-b7e3529339d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:44.020270 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95c777-ad25-4946-9078-b7e3529339d4 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:44.024223 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa83f2-5cea-4eba-835a-6843574a5b4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:44.024856 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa83f2-5cea-4eba-835a-6843574a5b4d service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:44.025136 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa83f2-5cea-4eba-835a-6843574a5b4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:44.025569 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa83f2-5cea-4eba-835a-6843574a5b4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:44.034782 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa83f2-5cea-4eba-835a-6843574a5b4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:44.036766 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa83f2-5cea-4eba-835a-6843574a5b4d service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:44.046334 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1204/2387] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:57:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:57:44.358286 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e870f3-a098-473c-aa5e-11a40444eb3c None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:44.358286 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a1e870f3-a098-473c-aa5e-11a40444eb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:44.370654 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e870f3-a098-473c-aa5e-11a40444eb3c None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:44.375480 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80635ec-a7f4-49a8-8675-38e18d33d972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:44.376150 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80635ec-a7f4-49a8-8675-38e18d33d972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:44.376427 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80635ec-a7f4-49a8-8675-38e18d33d972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:44.376697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80635ec-a7f4-49a8-8675-38e18d33d972 admin admin] PATH_INFO: `/v3/projects/61b1bfaf21e7460c92ac23b78f923945` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:44.377815 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80635ec-a7f4-49a8-8675-38e18d33d972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61b1bfaf21e7460c92ac23b78f923945)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:44.379810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80635ec-a7f4-49a8-8675-38e18d33d972 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:44.428108 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f80635ec-a7f4-49a8-8675-38e18d33d972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61b1bfaf21e7460c92ac23b78f923945', 'request_id': 'req-f80635ec-a7f4-49a8-8675-38e18d33d972'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:44.435615 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f80635ec-a7f4-49a8-8675-38e18d33d972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61b1bfaf21e7460c92ac23b78f923945', 'request_id': 'req-f80635ec-a7f4-49a8-8675-38e18d33d972'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:44.441323 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f80635ec-a7f4-49a8-8675-38e18d33d972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61b1bfaf21e7460c92ac23b78f923945', 'request_id': 'req-f80635ec-a7f4-49a8-8675-38e18d33d972'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:44.453153 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f80635ec-a7f4-49a8-8675-38e18d33d972 admin admin] The token cache is being invalidate because project 61b1bfaf21e7460c92ac23b78f923945 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:57:44.453442 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f80635ec-a7f4-49a8-8675-38e18d33d972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:44.454171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f80635ec-a7f4-49a8-8675-38e18d33d972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:44.455978 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1184/2388] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:57:44 2022] DELETE /identity/v3/projects/61b1bfaf21e7460c92ac23b78f923945 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:44.460451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa83f2-5cea-4eba-835a-6843574a5b4d service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:44.470272 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa83f2-5cea-4eba-835a-6843574a5b4d service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:44.527364 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82c2b5-2727-482e-b1ed-557f885a71fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:44.528025 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82c2b5-2727-482e-b1ed-557f885a71fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:44.528306 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82c2b5-2727-482e-b1ed-557f885a71fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:44.528579 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82c2b5-2727-482e-b1ed-557f885a71fe admin admin] PATH_INFO: `/v3/users/b8b9a024c3b942a6a188818f3e6c055f` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:44.529693 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82c2b5-2727-482e-b1ed-557f885a71fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8b9a024c3b942a6a188818f3e6c055f)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:44.531669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82c2b5-2727-482e-b1ed-557f885a71fe admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:44.572021 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:44.583819 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ff82c2b5-2727-482e-b1ed-557f885a71fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8b9a024c3b942a6a188818f3e6c055f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:44.595537 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ff82c2b5-2727-482e-b1ed-557f885a71fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8b9a024c3b942a6a188818f3e6c055f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:44.604039 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ff82c2b5-2727-482e-b1ed-557f885a71fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8b9a024c3b942a6a188818f3e6c055f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:44.604661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ff82c2b5-2727-482e-b1ed-557f885a71fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8b9a024c3b942a6a188818f3e6c055f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:44.611524 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ff82c2b5-2727-482e-b1ed-557f885a71fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8b9a024c3b942a6a188818f3e6c055f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:44.617715 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1205/2389] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:57:44 2022] DELETE /identity/v3/users/b8b9a024c3b942a6a188818f3e6c055f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:44.892252 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80635ec-a7f4-49a8-8675-38e18d33d972 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:44.893723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-f80635ec-a7f4-49a8-8675-38e18d33d972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:44.909478 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80635ec-a7f4-49a8-8675-38e18d33d972 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:44.958371 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:44.976629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450bf49-9825-4bf3-97c8-fb424559ad8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:44.977262 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450bf49-9825-4bf3-97c8-fb424559ad8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:44.977541 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450bf49-9825-4bf3-97c8-fb424559ad8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:44.977816 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450bf49-9825-4bf3-97c8-fb424559ad8b admin admin] PATH_INFO: `/v3/projects/35c132ce49124f0e8866510cf8f5e890` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:44.978939 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450bf49-9825-4bf3-97c8-fb424559ad8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=35c132ce49124f0e8866510cf8f5e890)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:44.980910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450bf49-9825-4bf3-97c8-fb424559ad8b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:45.028893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0450bf49-9825-4bf3-97c8-fb424559ad8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35c132ce49124f0e8866510cf8f5e890', 'request_id': 'req-0450bf49-9825-4bf3-97c8-fb424559ad8b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:45.037175 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0450bf49-9825-4bf3-97c8-fb424559ad8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35c132ce49124f0e8866510cf8f5e890', 'request_id': 'req-0450bf49-9825-4bf3-97c8-fb424559ad8b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:45.042997 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0450bf49-9825-4bf3-97c8-fb424559ad8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35c132ce49124f0e8866510cf8f5e890', 'request_id': 'req-0450bf49-9825-4bf3-97c8-fb424559ad8b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:45.054564 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0450bf49-9825-4bf3-97c8-fb424559ad8b admin admin] The token cache is being invalidate because project 35c132ce49124f0e8866510cf8f5e890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:57:45.054861 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0450bf49-9825-4bf3-97c8-fb424559ad8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:45.055544 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0450bf49-9825-4bf3-97c8-fb424559ad8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:45.057414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1185/2390] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:57:44 2022] DELETE /identity/v3/projects/35c132ce49124f0e8866510cf8f5e890 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:45.062147 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51117531-4165-449c-91b7-de210a95311b None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:45.062613 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51117531-4165-449c-91b7-de210a95311b None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:45.062784 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51117531-4165-449c-91b7-de210a95311b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:45.082003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-51117531-4165-449c-91b7-de210a95311b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:45.083745 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-51117531-4165-449c-91b7-de210a95311b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:45.141735 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1206/2391] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:45.149861 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c9cd8-27bf-46d0-bbac-c0d55f866755 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:45.150161 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c9cd8-27bf-46d0-bbac-c0d55f866755 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:45.150437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c9cd8-27bf-46d0-bbac-c0d55f866755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:45.169919 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-547c9cd8-27bf-46d0-bbac-c0d55f866755 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:45.171110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-547c9cd8-27bf-46d0-bbac-c0d55f866755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:45.178910 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1186/2392] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:45.182954 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51117531-4165-449c-91b7-de210a95311b None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:45.192950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51117531-4165-449c-91b7-de210a95311b None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:45.197424 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea215a6-3105-4dcc-a162-fd175046eb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:45.198006 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea215a6-3105-4dcc-a162-fd175046eb33 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:45.198280 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea215a6-3105-4dcc-a162-fd175046eb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:45.198545 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea215a6-3105-4dcc-a162-fd175046eb33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:45.199191 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea215a6-3105-4dcc-a162-fd175046eb33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:57:45.199484 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea215a6-3105-4dcc-a162-fd175046eb33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:45.201095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea215a6-3105-4dcc-a162-fd175046eb33 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:45.212986 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1207/2393] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:57:45.216986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c9cd8-27bf-46d0-bbac-c0d55f866755 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:45.226031 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c9cd8-27bf-46d0-bbac-c0d55f866755 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:45.230633 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4f465-4664-4b22-a4ed-eb6ca2182357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:45.231221 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4f465-4664-4b22-a4ed-eb6ca2182357 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:45.231499 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4f465-4664-4b22-a4ed-eb6ca2182357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:45.231821 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4f465-4664-4b22-a4ed-eb6ca2182357 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:45.232430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4f465-4664-4b22-a4ed-eb6ca2182357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:45.234298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4f465-4664-4b22-a4ed-eb6ca2182357 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:45.261735 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1187/2394] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:57:45 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:57:45.265756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea215a6-3105-4dcc-a162-fd175046eb33 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:45.276476 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea215a6-3105-4dcc-a162-fd175046eb33 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:45.332260 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8c0b4-be8d-47ab-995f-813a44411ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:45.333432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8c0b4-be8d-47ab-995f-813a44411ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:45.333705 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8c0b4-be8d-47ab-995f-813a44411ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:45.334062 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8c0b4-be8d-47ab-995f-813a44411ea9 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:45.334576 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8c0b4-be8d-47ab-995f-813a44411ea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:45.336678 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8c0b4-be8d-47ab-995f-813a44411ea9 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:45.358895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1208/2395] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:57:45 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:57:45.362967 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4f465-4664-4b22-a4ed-eb6ca2182357 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:45.373127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4f465-4664-4b22-a4ed-eb6ca2182357 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:45.378063 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1bd227-353d-4685-a857-e187af9bc039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:45.378664 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bd227-353d-4685-a857-e187af9bc039 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:45.378943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bd227-353d-4685-a857-e187af9bc039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:45.379240 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bd227-353d-4685-a857-e187af9bc039 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:45.380414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1bd227-353d-4685-a857-e187af9bc039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:45.382005 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1bd227-353d-4685-a857-e187af9bc039 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:45.392128 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1188/2396] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:57:45.396320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8c0b4-be8d-47ab-995f-813a44411ea9 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:45.405971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8c0b4-be8d-47ab-995f-813a44411ea9 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:45.410507 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e7912-c86a-4a7f-8512-9112e708c1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:45.411144 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e7912-c86a-4a7f-8512-9112e708c1bf admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:45.411423 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e7912-c86a-4a7f-8512-9112e708c1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:45.411736 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e7912-c86a-4a7f-8512-9112e708c1bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:45.412366 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e7912-c86a-4a7f-8512-9112e708c1bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:45.413945 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e7912-c86a-4a7f-8512-9112e708c1bf admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:45.424250 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1209/2397] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:57:45.428076 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1bd227-353d-4685-a857-e187af9bc039 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:45.437750 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1bd227-353d-4685-a857-e187af9bc039 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:45.442888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e6ec7-95ac-4b21-8fbb-98c213d47ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:45.443523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e6ec7-95ac-4b21-8fbb-98c213d47ced admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:45.443830 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e6ec7-95ac-4b21-8fbb-98c213d47ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:45.444100 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e6ec7-95ac-4b21-8fbb-98c213d47ced admin admin] PATH_INFO: `/v3/projects/5dd5bbb96ed3423ca9c18aae6af7f10e/users/5be56056fdc2491da9e39399609ded79/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:45.459263 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e6ec7-95ac-4b21-8fbb-98c213d47ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dd5bbb96ed3423ca9c18aae6af7f10e, user_id=5be56056fdc2491da9e39399609ded79, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:45.461961 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e6ec7-95ac-4b21-8fbb-98c213d47ced admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:45.474949 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1189/2398] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:57:45 2022] PUT /identity/v3/projects/5dd5bbb96ed3423ca9c18aae6af7f10e/users/5be56056fdc2491da9e39399609ded79/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:45.488556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530de2c2-5190-4178-8750-9645499ae84b None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:45.489348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530de2c2-5190-4178-8750-9645499ae84b None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:45.489646 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530de2c2-5190-4178-8750-9645499ae84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:45.517258 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-530de2c2-5190-4178-8750-9645499ae84b None None] MFA Rules not processed for user `5be56056fdc2491da9e39399609ded79`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:45.518449 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-530de2c2-5190-4178-8750-9645499ae84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:45.523325 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:45.582347 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1210/2399] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:45.586697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e6ec7-95ac-4b21-8fbb-98c213d47ced admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:45.647202 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e6ec7-95ac-4b21-8fbb-98c213d47ced admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:45.651743 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc326fd8-9483-4b7a-9fd9-73365d4bfd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:45.652331 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc326fd8-9483-4b7a-9fd9-73365d4bfd22 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:45.652603 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc326fd8-9483-4b7a-9fd9-73365d4bfd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:45.652870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc326fd8-9483-4b7a-9fd9-73365d4bfd22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:45.653496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc326fd8-9483-4b7a-9fd9-73365d4bfd22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:45.655400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc326fd8-9483-4b7a-9fd9-73365d4bfd22 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:45.669665 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:45.682781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1190/2400] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:57:45 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:57:45.686780 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530de2c2-5190-4178-8750-9645499ae84b None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:45.697728 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530de2c2-5190-4178-8750-9645499ae84b None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:45.752448 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827b54c-3bdf-40ca-aa2d-cdac5bb1dbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:45.753053 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827b54c-3bdf-40ca-aa2d-cdac5bb1dbce admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:45.753332 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827b54c-3bdf-40ca-aa2d-cdac5bb1dbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:45.753602 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827b54c-3bdf-40ca-aa2d-cdac5bb1dbce admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:45.754227 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827b54c-3bdf-40ca-aa2d-cdac5bb1dbce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:45.756839 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827b54c-3bdf-40ca-aa2d-cdac5bb1dbce admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:45.777906 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1211/2401] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:57:45 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:57:45.781855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc326fd8-9483-4b7a-9fd9-73365d4bfd22 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:45.791872 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc326fd8-9483-4b7a-9fd9-73365d4bfd22 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:45.796627 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69079739-373e-4a76-9cb4-85900312d0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:45.797229 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69079739-373e-4a76-9cb4-85900312d0f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:45.797508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69079739-373e-4a76-9cb4-85900312d0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:45.797808 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69079739-373e-4a76-9cb4-85900312d0f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:45.798441 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69079739-373e-4a76-9cb4-85900312d0f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:45.800092 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69079739-373e-4a76-9cb4-85900312d0f7 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:45.811102 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1191/2402] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:57:45.815069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827b54c-3bdf-40ca-aa2d-cdac5bb1dbce admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:45.825359 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827b54c-3bdf-40ca-aa2d-cdac5bb1dbce admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:45.829905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe56064-9fcf-4bb0-8968-b5f828417e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:45.830546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe56064-9fcf-4bb0-8968-b5f828417e34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:45.830821 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe56064-9fcf-4bb0-8968-b5f828417e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:45.831090 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe56064-9fcf-4bb0-8968-b5f828417e34 admin admin] PATH_INFO: `/v3/projects/fd4794cddc1e4a1d84406d0f0f585dcf/users/0b010323264b47fe95e40fdeee83977d/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:45.845730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe56064-9fcf-4bb0-8968-b5f828417e34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd4794cddc1e4a1d84406d0f0f585dcf, user_id=0b010323264b47fe95e40fdeee83977d, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:45.848900 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe56064-9fcf-4bb0-8968-b5f828417e34 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:45.861719 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1212/2403] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:57:45 2022] PUT /identity/v3/projects/fd4794cddc1e4a1d84406d0f0f585dcf/users/0b010323264b47fe95e40fdeee83977d/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:45.866207 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69079739-373e-4a76-9cb4-85900312d0f7 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:45.876006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69079739-373e-4a76-9cb4-85900312d0f7 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:45.931268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbe430-26a1-41a6-a75a-e8f3f0676c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:45.931885 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbe430-26a1-41a6-a75a-e8f3f0676c83 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:45.932161 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbe430-26a1-41a6-a75a-e8f3f0676c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:45.932429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbe430-26a1-41a6-a75a-e8f3f0676c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:45.933052 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbe430-26a1-41a6-a75a-e8f3f0676c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:45.934619 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbe430-26a1-41a6-a75a-e8f3f0676c83 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:45.944072 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1192/2404] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:57:45.948096 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe56064-9fcf-4bb0-8968-b5f828417e34 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:45.958320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe56064-9fcf-4bb0-8968-b5f828417e34 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:45.962768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c720b-1a43-4c68-96bd-f99033a404af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:45.963402 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c720b-1a43-4c68-96bd-f99033a404af admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:45.963704 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c720b-1a43-4c68-96bd-f99033a404af admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:45.963970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c720b-1a43-4c68-96bd-f99033a404af admin admin] PATH_INFO: `/v3/projects/fd4794cddc1e4a1d84406d0f0f585dcf/users/0b010323264b47fe95e40fdeee83977d/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:45.965844 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c720b-1a43-4c68-96bd-f99033a404af admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd4794cddc1e4a1d84406d0f0f585dcf, user_id=0b010323264b47fe95e40fdeee83977d, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:45.968497 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c720b-1a43-4c68-96bd-f99033a404af admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:45.978881 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1213/2405] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:57:45 2022] PUT /identity/v3/projects/fd4794cddc1e4a1d84406d0f0f585dcf/users/0b010323264b47fe95e40fdeee83977d/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:45.989422 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1de8ba-3518-406a-b46e-9d12797a9a96 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:45.989708 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1de8ba-3518-406a-b46e-9d12797a9a96 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:45.989978 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1de8ba-3518-406a-b46e-9d12797a9a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:46.016817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-3e1de8ba-3518-406a-b46e-9d12797a9a96 None None] MFA Rules not processed for user `0b010323264b47fe95e40fdeee83977d`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:46.018005 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3e1de8ba-3518-406a-b46e-9d12797a9a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:46.084593 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1193/2406] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:46.095677 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c720b-1a43-4c68-96bd-f99033a404af admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:46.097434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-015c720b-1a43-4c68-96bd-f99033a404af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:46.158987 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c720b-1a43-4c68-96bd-f99033a404af admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:46.163520 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6360fe-d570-485e-9a4e-5b0a0cf3a6db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:46.163746 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6360fe-d570-485e-9a4e-5b0a0cf3a6db service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:46.164018 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6360fe-d570-485e-9a4e-5b0a0cf3a6db service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:46.164423 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6360fe-d570-485e-9a4e-5b0a0cf3a6db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:46.173539 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6360fe-d570-485e-9a4e-5b0a0cf3a6db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:46.174996 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6360fe-d570-485e-9a4e-5b0a0cf3a6db service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:46.186625 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1214/2407] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:57:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:57:46.252477 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1de8ba-3518-406a-b46e-9d12797a9a96 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:46.312482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1de8ba-3518-406a-b46e-9d12797a9a96 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:46.317131 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba0d88-df98-495d-b708-55b7fd585a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:46.317759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba0d88-df98-495d-b708-55b7fd585a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:46.318040 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba0d88-df98-495d-b708-55b7fd585a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:46.318311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba0d88-df98-495d-b708-55b7fd585a6f admin admin] PATH_INFO: `/v3/users/5be56056fdc2491da9e39399609ded79` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:46.319458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba0d88-df98-495d-b708-55b7fd585a6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5be56056fdc2491da9e39399609ded79)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:46.321471 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba0d88-df98-495d-b708-55b7fd585a6f admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:46.326721 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:46.368258 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-20ba0d88-df98-495d-b708-55b7fd585a6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5be56056fdc2491da9e39399609ded79'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:46.368904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-20ba0d88-df98-495d-b708-55b7fd585a6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5be56056fdc2491da9e39399609ded79'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:46.381066 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-20ba0d88-df98-495d-b708-55b7fd585a6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5be56056fdc2491da9e39399609ded79'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:46.388107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-20ba0d88-df98-495d-b708-55b7fd585a6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5be56056fdc2491da9e39399609ded79'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:46.388738 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-20ba0d88-df98-495d-b708-55b7fd585a6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5be56056fdc2491da9e39399609ded79'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:46.402102 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1194/2408] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:57:46 2022] DELETE /identity/v3/users/5be56056fdc2491da9e39399609ded79 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:46.406427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d9f65-a2c2-4b88-8bdb-828ceafb1a32 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:46.406710 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d9f65-a2c2-4b88-8bdb-828ceafb1a32 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:46.406978 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d9f65-a2c2-4b88-8bdb-828ceafb1a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:46.414792 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:46.427646 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-7f5d9f65-a2c2-4b88-8bdb-828ceafb1a32 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:46.428830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-7f5d9f65-a2c2-4b88-8bdb-828ceafb1a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:46.485968 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1215/2409] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:57:46.495718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba0d88-df98-495d-b708-55b7fd585a6f admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:46.497116 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-20ba0d88-df98-495d-b708-55b7fd585a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:46.552998 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba0d88-df98-495d-b708-55b7fd585a6f admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:46.556642 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faca3c2-e1a2-4ed8-ac0f-665b4ada1942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:46.557236 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faca3c2-e1a2-4ed8-ac0f-665b4ada1942 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:46.557514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faca3c2-e1a2-4ed8-ac0f-665b4ada1942 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:46.557784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faca3c2-e1a2-4ed8-ac0f-665b4ada1942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:46.566493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faca3c2-e1a2-4ed8-ac0f-665b4ada1942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:46.567956 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faca3c2-e1a2-4ed8-ac0f-665b4ada1942 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:46.577061 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1195/2410] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:57:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:57:46.841119 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d9f65-a2c2-4b88-8bdb-828ceafb1a32 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:46.842705 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-7f5d9f65-a2c2-4b88-8bdb-828ceafb1a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:46.856323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d9f65-a2c2-4b88-8bdb-828ceafb1a32 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:46.861343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595adedb-c3e0-4cfc-ba92-b4969d8f5720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:46.861975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595adedb-c3e0-4cfc-ba92-b4969d8f5720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:46.862245 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595adedb-c3e0-4cfc-ba92-b4969d8f5720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:46.862514 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595adedb-c3e0-4cfc-ba92-b4969d8f5720 admin admin] PATH_INFO: `/v3/projects/5dd5bbb96ed3423ca9c18aae6af7f10e` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:46.863641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595adedb-c3e0-4cfc-ba92-b4969d8f5720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dd5bbb96ed3423ca9c18aae6af7f10e)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:46.865641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595adedb-c3e0-4cfc-ba92-b4969d8f5720 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:46.912068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-595adedb-c3e0-4cfc-ba92-b4969d8f5720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dd5bbb96ed3423ca9c18aae6af7f10e', 'request_id': 'req-595adedb-c3e0-4cfc-ba92-b4969d8f5720'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:46.918143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-595adedb-c3e0-4cfc-ba92-b4969d8f5720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dd5bbb96ed3423ca9c18aae6af7f10e', 'request_id': 'req-595adedb-c3e0-4cfc-ba92-b4969d8f5720'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:46.925043 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-595adedb-c3e0-4cfc-ba92-b4969d8f5720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dd5bbb96ed3423ca9c18aae6af7f10e', 'request_id': 'req-595adedb-c3e0-4cfc-ba92-b4969d8f5720'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:46.935071 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-595adedb-c3e0-4cfc-ba92-b4969d8f5720 admin admin] The token cache is being invalidate because project 5dd5bbb96ed3423ca9c18aae6af7f10e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:57:46.935354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-595adedb-c3e0-4cfc-ba92-b4969d8f5720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:46.935996 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-595adedb-c3e0-4cfc-ba92-b4969d8f5720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:46.937477 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1216/2411] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:57:46 2022] DELETE /identity/v3/projects/5dd5bbb96ed3423ca9c18aae6af7f10e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:46.941545 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faca3c2-e1a2-4ed8-ac0f-665b4ada1942 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:46.951541 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faca3c2-e1a2-4ed8-ac0f-665b4ada1942 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:47.007333 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62dbbde-375d-46a9-a033-c5888a0ae84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:47.007990 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62dbbde-375d-46a9-a033-c5888a0ae84c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:47.008265 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62dbbde-375d-46a9-a033-c5888a0ae84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:47.008535 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62dbbde-375d-46a9-a033-c5888a0ae84c admin admin] PATH_INFO: `/v3/users/0b010323264b47fe95e40fdeee83977d` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:47.009881 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62dbbde-375d-46a9-a033-c5888a0ae84c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b010323264b47fe95e40fdeee83977d)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:47.011837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62dbbde-375d-46a9-a033-c5888a0ae84c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:47.043374 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:47.057747 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b62dbbde-375d-46a9-a033-c5888a0ae84c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b010323264b47fe95e40fdeee83977d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:47.058488 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b62dbbde-375d-46a9-a033-c5888a0ae84c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b010323264b47fe95e40fdeee83977d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:47.071094 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b62dbbde-375d-46a9-a033-c5888a0ae84c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b010323264b47fe95e40fdeee83977d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:47.077504 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b62dbbde-375d-46a9-a033-c5888a0ae84c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b010323264b47fe95e40fdeee83977d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:47.078126 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-b62dbbde-375d-46a9-a033-c5888a0ae84c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b010323264b47fe95e40fdeee83977d'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:47.090050 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1196/2412] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:57:46 2022] DELETE /identity/v3/users/0b010323264b47fe95e40fdeee83977d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:47.449934 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595adedb-c3e0-4cfc-ba92-b4969d8f5720 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:47.452108 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-595adedb-c3e0-4cfc-ba92-b4969d8f5720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:47.469659 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595adedb-c3e0-4cfc-ba92-b4969d8f5720 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:47.549420 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aef94d-97f1-45cf-88a8-a18f6278fee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:47.550085 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aef94d-97f1-45cf-88a8-a18f6278fee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:47.550351 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aef94d-97f1-45cf-88a8-a18f6278fee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:47.550621 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aef94d-97f1-45cf-88a8-a18f6278fee7 admin admin] PATH_INFO: `/v3/projects/fd4794cddc1e4a1d84406d0f0f585dcf` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:47.552387 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aef94d-97f1-45cf-88a8-a18f6278fee7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd4794cddc1e4a1d84406d0f0f585dcf)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:47.554353 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aef94d-97f1-45cf-88a8-a18f6278fee7 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:47.587693 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:47.606316 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-a1aef94d-97f1-45cf-88a8-a18f6278fee7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd4794cddc1e4a1d84406d0f0f585dcf', 'request_id': 'req-a1aef94d-97f1-45cf-88a8-a18f6278fee7'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:47.613058 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-a1aef94d-97f1-45cf-88a8-a18f6278fee7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd4794cddc1e4a1d84406d0f0f585dcf', 'request_id': 'req-a1aef94d-97f1-45cf-88a8-a18f6278fee7'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:47.621809 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-a1aef94d-97f1-45cf-88a8-a18f6278fee7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd4794cddc1e4a1d84406d0f0f585dcf', 'request_id': 'req-a1aef94d-97f1-45cf-88a8-a18f6278fee7'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:47.633018 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-a1aef94d-97f1-45cf-88a8-a18f6278fee7 admin admin] The token cache is being invalidate because project fd4794cddc1e4a1d84406d0f0f585dcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:57:47.633446 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-a1aef94d-97f1-45cf-88a8-a18f6278fee7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:47.634368 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-a1aef94d-97f1-45cf-88a8-a18f6278fee7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:57:47.635925 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1217/2413] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:57:47 2022] DELETE /identity/v3/projects/fd4794cddc1e4a1d84406d0f0f585dcf => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:57:47.641000 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3a099-6eb2-48dc-99c1-7e28ae680369 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:47.641288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3a099-6eb2-48dc-99c1-7e28ae680369 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:47.641560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3a099-6eb2-48dc-99c1-7e28ae680369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:47.664090 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-ddc3a099-6eb2-48dc-99c1-7e28ae680369 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:47.665330 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ddc3a099-6eb2-48dc-99c1-7e28ae680369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:47.745147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1197/2414] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:57:47.753410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5f509-e999-40a0-ad52-e62b82ff2919 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:47.753699 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5f509-e999-40a0-ad52-e62b82ff2919 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:47.753975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5f509-e999-40a0-ad52-e62b82ff2919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:47.775448 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-9ec5f509-e999-40a0-ad52-e62b82ff2919 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:47.777051 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9ec5f509-e999-40a0-ad52-e62b82ff2919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:47.787748 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1218/2415] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:47.789765 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3a099-6eb2-48dc-99c1-7e28ae680369 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:47.799591 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3a099-6eb2-48dc-99c1-7e28ae680369 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:47.804781 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefaf195-6c5c-42a6-9ff8-9fe458f4f53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:47.805378 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefaf195-6c5c-42a6-9ff8-9fe458f4f53b admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:47.805654 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefaf195-6c5c-42a6-9ff8-9fe458f4f53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:47.805919 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefaf195-6c5c-42a6-9ff8-9fe458f4f53b admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:47.806557 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefaf195-6c5c-42a6-9ff8-9fe458f4f53b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:57:47.806854 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefaf195-6c5c-42a6-9ff8-9fe458f4f53b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:47.808449 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefaf195-6c5c-42a6-9ff8-9fe458f4f53b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:47.824566 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1198/2416] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:57:47.826134 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5f509-e999-40a0-ad52-e62b82ff2919 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:47.835674 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5f509-e999-40a0-ad52-e62b82ff2919 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:47.840546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874e15c-3aaf-47f9-81a7-ef14f28dc853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:47.841221 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874e15c-3aaf-47f9-81a7-ef14f28dc853 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:47.841556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874e15c-3aaf-47f9-81a7-ef14f28dc853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:47.841891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874e15c-3aaf-47f9-81a7-ef14f28dc853 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:47.842834 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874e15c-3aaf-47f9-81a7-ef14f28dc853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:47.844797 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874e15c-3aaf-47f9-81a7-ef14f28dc853 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:47.874125 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1219/2417] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:57:47 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:57:47.878499 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefaf195-6c5c-42a6-9ff8-9fe458f4f53b admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:47.890409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefaf195-6c5c-42a6-9ff8-9fe458f4f53b admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:47.949271 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875384f-875c-49ce-957b-c1f9b9393980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:47.950040 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875384f-875c-49ce-957b-c1f9b9393980 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:47.950448 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875384f-875c-49ce-957b-c1f9b9393980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:47.950814 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875384f-875c-49ce-957b-c1f9b9393980 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:47.951547 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875384f-875c-49ce-957b-c1f9b9393980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:47.953590 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875384f-875c-49ce-957b-c1f9b9393980 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:47.979733 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1199/2418] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:57:47 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:57:47.983163 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874e15c-3aaf-47f9-81a7-ef14f28dc853 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:47.994898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874e15c-3aaf-47f9-81a7-ef14f28dc853 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:47.999773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584881a7-4809-4665-9efc-49522d6684ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:48.000388 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584881a7-4809-4665-9efc-49522d6684ce admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:48.000669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584881a7-4809-4665-9efc-49522d6684ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:48.001003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584881a7-4809-4665-9efc-49522d6684ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:48.001646 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584881a7-4809-4665-9efc-49522d6684ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:48.003267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584881a7-4809-4665-9efc-49522d6684ce admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:48.017133 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1220/2419] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:57:48.018373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875384f-875c-49ce-957b-c1f9b9393980 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:48.029855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875384f-875c-49ce-957b-c1f9b9393980 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:48.037473 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36f34b-0a6f-44a9-8692-cac676da2b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:48.038098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36f34b-0a6f-44a9-8692-cac676da2b70 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:48.038474 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36f34b-0a6f-44a9-8692-cac676da2b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:48.038647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36f34b-0a6f-44a9-8692-cac676da2b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:48.039339 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36f34b-0a6f-44a9-8692-cac676da2b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:48.041014 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36f34b-0a6f-44a9-8692-cac676da2b70 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:48.052795 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1200/2420] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:57:48.057050 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584881a7-4809-4665-9efc-49522d6684ce admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:48.067983 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584881a7-4809-4665-9efc-49522d6684ce admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:48.073656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbe219-8e2c-4c74-8286-6603cba88d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:48.074332 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbe219-8e2c-4c74-8286-6603cba88d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:48.074658 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbe219-8e2c-4c74-8286-6603cba88d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:48.074935 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbe219-8e2c-4c74-8286-6603cba88d53 admin admin] PATH_INFO: `/v3/projects/f1e08a0eb70b439c89cce584b26b1cd8/users/326b7d95610145c2ab97b64a876782f3/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:48.091025 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbe219-8e2c-4c74-8286-6603cba88d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1e08a0eb70b439c89cce584b26b1cd8, user_id=326b7d95610145c2ab97b64a876782f3, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:48.093772 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbe219-8e2c-4c74-8286-6603cba88d53 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:48.106949 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1221/2421] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:57:48 2022] PUT /identity/v3/projects/f1e08a0eb70b439c89cce584b26b1cd8/users/326b7d95610145c2ab97b64a876782f3/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:57:48.118470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05845ff8-1a17-4a6b-8206-c0d9a6cd61b7 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:48.118756 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05845ff8-1a17-4a6b-8206-c0d9a6cd61b7 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:48.119031 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05845ff8-1a17-4a6b-8206-c0d9a6cd61b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:48.144590 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:48.148419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-05845ff8-1a17-4a6b-8206-c0d9a6cd61b7 None None] MFA Rules not processed for user `326b7d95610145c2ab97b64a876782f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:57:48.150188 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-05845ff8-1a17-4a6b-8206-c0d9a6cd61b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:48.213578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1201/2422] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:57:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:57:48.224765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbe219-8e2c-4c74-8286-6603cba88d53 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:57:48.226442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-65bbe219-8e2c-4c74-8286-6603cba88d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:57:48.281279 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbe219-8e2c-4c74-8286-6603cba88d53 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:57:48.284765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2ff35-55bd-46aa-838e-5e86e5efa778 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:57:48.285360 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2ff35-55bd-46aa-838e-5e86e5efa778 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:57:48.286137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2ff35-55bd-46aa-838e-5e86e5efa778 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:57:48.286397 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2ff35-55bd-46aa-838e-5e86e5efa778 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:57:48.294477 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2ff35-55bd-46aa-838e-5e86e5efa778 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:57:48.295921 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2ff35-55bd-46aa-838e-5e86e5efa778 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:57:48.304370 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1222/2423] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:57:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:57:58.210640 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=26 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:57:58.300507 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=67,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:04.561803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e019474-828e-46ca-a078-987a60aba247 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:04.562227 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e019474-828e-46ca-a078-987a60aba247 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:04.562354 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e019474-828e-46ca-a078-987a60aba247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:04.582137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-3e019474-828e-46ca-a078-987a60aba247 None None] MFA Rules not processed for user `a665b883f5ab4c408329c358da8972cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:58:04.583320 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3e019474-828e-46ca-a078-987a60aba247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:04.639598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1202/2424] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:58:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:58:14.632522 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=25 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:15.182390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2ff35-55bd-46aa-838e-5e86e5efa778 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:15.242996 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2ff35-55bd-46aa-838e-5e86e5efa778 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:15.247437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeec88e-a718-4479-a447-6e1c6233f974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:15.248077 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeec88e-a718-4479-a447-6e1c6233f974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:15.248353 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeec88e-a718-4479-a447-6e1c6233f974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:15.248618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeec88e-a718-4479-a447-6e1c6233f974 admin admin] PATH_INFO: `/v3/users/326b7d95610145c2ab97b64a876782f3` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:15.249708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeec88e-a718-4479-a447-6e1c6233f974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=326b7d95610145c2ab97b64a876782f3)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:15.251630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeec88e-a718-4479-a447-6e1c6233f974 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:15.297517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-eeeec88e-a718-4479-a447-6e1c6233f974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '326b7d95610145c2ab97b64a876782f3'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:15.309391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-eeeec88e-a718-4479-a447-6e1c6233f974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '326b7d95610145c2ab97b64a876782f3'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:15.316685 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-eeeec88e-a718-4479-a447-6e1c6233f974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '326b7d95610145c2ab97b64a876782f3'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:15.317286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-eeeec88e-a718-4479-a447-6e1c6233f974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '326b7d95610145c2ab97b64a876782f3'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:15.324458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-eeeec88e-a718-4479-a447-6e1c6233f974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '326b7d95610145c2ab97b64a876782f3'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:15.330165 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1223/2425] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:58:15 2022] DELETE /identity/v3/users/326b7d95610145c2ab97b64a876782f3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:58:15.335186 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ee13e-5b2b-4a6f-b004-5b48ae5dab30 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:15.335467 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ee13e-5b2b-4a6f-b004-5b48ae5dab30 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:15.335766 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ee13e-5b2b-4a6f-b004-5b48ae5dab30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:15.354409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-939ee13e-5b2b-4a6f-b004-5b48ae5dab30 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:58:15.355676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-939ee13e-5b2b-4a6f-b004-5b48ae5dab30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:15.412670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1203/2426] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:58:15.422202 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeec88e-a718-4479-a447-6e1c6233f974 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:15.423649 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-eeeec88e-a718-4479-a447-6e1c6233f974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:15.478772 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeec88e-a718-4479-a447-6e1c6233f974 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:15.482317 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e6a95-2d53-4425-8f43-16bd32069250 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:15.482905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e6a95-2d53-4425-8f43-16bd32069250 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:15.483179 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e6a95-2d53-4425-8f43-16bd32069250 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:15.483443 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e6a95-2d53-4425-8f43-16bd32069250 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:15.493911 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e6a95-2d53-4425-8f43-16bd32069250 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:15.495474 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e6a95-2d53-4425-8f43-16bd32069250 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:15.506463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1224/2427] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:58:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:58:15.801755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ee13e-5b2b-4a6f-b004-5b48ae5dab30 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:15.803997 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-939ee13e-5b2b-4a6f-b004-5b48ae5dab30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:15.815858 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ee13e-5b2b-4a6f-b004-5b48ae5dab30 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:15.821605 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6761c0-c9ee-42d4-9aab-8dc48483f807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:15.822317 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6761c0-c9ee-42d4-9aab-8dc48483f807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:15.822596 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6761c0-c9ee-42d4-9aab-8dc48483f807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:15.822893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6761c0-c9ee-42d4-9aab-8dc48483f807 admin admin] PATH_INFO: `/v3/projects/f1e08a0eb70b439c89cce584b26b1cd8` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:15.824015 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6761c0-c9ee-42d4-9aab-8dc48483f807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1e08a0eb70b439c89cce584b26b1cd8)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:15.826784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6761c0-c9ee-42d4-9aab-8dc48483f807 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:15.874448 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1e6761c0-c9ee-42d4-9aab-8dc48483f807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1e08a0eb70b439c89cce584b26b1cd8', 'request_id': 'req-1e6761c0-c9ee-42d4-9aab-8dc48483f807'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:15.881268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1e6761c0-c9ee-42d4-9aab-8dc48483f807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1e08a0eb70b439c89cce584b26b1cd8', 'request_id': 'req-1e6761c0-c9ee-42d4-9aab-8dc48483f807'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:15.887429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1e6761c0-c9ee-42d4-9aab-8dc48483f807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1e08a0eb70b439c89cce584b26b1cd8', 'request_id': 'req-1e6761c0-c9ee-42d4-9aab-8dc48483f807'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:15.898719 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1e6761c0-c9ee-42d4-9aab-8dc48483f807 admin admin] The token cache is being invalidate because project f1e08a0eb70b439c89cce584b26b1cd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:58:15.899007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1e6761c0-c9ee-42d4-9aab-8dc48483f807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:15.899677 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-1e6761c0-c9ee-42d4-9aab-8dc48483f807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:15.901160 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1204/2428] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:58:15 2022] DELETE /identity/v3/projects/f1e08a0eb70b439c89cce584b26b1cd8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:58:15.906129 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85dcc0-89ae-4357-8a48-29bbc5d8d7a5 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:15.906446 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85dcc0-89ae-4357-8a48-29bbc5d8d7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:15.906715 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85dcc0-89ae-4357-8a48-29bbc5d8d7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:15.926033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-fb85dcc0-89ae-4357-8a48-29bbc5d8d7a5 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:58:15.927208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-fb85dcc0-89ae-4357-8a48-29bbc5d8d7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:15.943823 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:15.986348 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1225/2429] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 21:58:15.994773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cdd0b-02a3-4e3b-8190-d4a576bdd58d None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:15.995081 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cdd0b-02a3-4e3b-8190-d4a576bdd58d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:15.995355 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cdd0b-02a3-4e3b-8190-d4a576bdd58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:16.014719 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-8f7cdd0b-02a3-4e3b-8190-d4a576bdd58d None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:58:16.015929 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8f7cdd0b-02a3-4e3b-8190-d4a576bdd58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:16.023712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1205/2430] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:58:16.027712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85dcc0-89ae-4357-8a48-29bbc5d8d7a5 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:16.037603 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85dcc0-89ae-4357-8a48-29bbc5d8d7a5 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:16.042019 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ef2e1-6503-4cd3-ad90-72a39539b054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:16.042608 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ef2e1-6503-4cd3-ad90-72a39539b054 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:16.042889 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ef2e1-6503-4cd3-ad90-72a39539b054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:16.043157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ef2e1-6503-4cd3-ad90-72a39539b054 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:16.043810 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ef2e1-6503-4cd3-ad90-72a39539b054 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:58:16.044109 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ef2e1-6503-4cd3-ad90-72a39539b054 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:16.045692 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ef2e1-6503-4cd3-ad90-72a39539b054 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:16.057406 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1226/2431] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:58:16.061585 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cdd0b-02a3-4e3b-8190-d4a576bdd58d None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:16.071139 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cdd0b-02a3-4e3b-8190-d4a576bdd58d None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:16.075886 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877130c0-828d-43ef-a6a5-c79c7002baf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:16.076483 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877130c0-828d-43ef-a6a5-c79c7002baf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:16.076758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877130c0-828d-43ef-a6a5-c79c7002baf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:16.077078 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877130c0-828d-43ef-a6a5-c79c7002baf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:16.077653 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877130c0-828d-43ef-a6a5-c79c7002baf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:16.079536 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877130c0-828d-43ef-a6a5-c79c7002baf7 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:16.111729 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1206/2432] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:58:16 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:58:16.117175 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ef2e1-6503-4cd3-ad90-72a39539b054 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:16.126985 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ef2e1-6503-4cd3-ad90-72a39539b054 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:16.181086 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc780feb-76aa-41ec-a6e7-fcbfa94a4042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:16.181696 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc780feb-76aa-41ec-a6e7-fcbfa94a4042 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:16.181979 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc780feb-76aa-41ec-a6e7-fcbfa94a4042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:16.182253 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc780feb-76aa-41ec-a6e7-fcbfa94a4042 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:16.182873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc780feb-76aa-41ec-a6e7-fcbfa94a4042 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:16.184831 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc780feb-76aa-41ec-a6e7-fcbfa94a4042 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:16.211588 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1227/2433] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:58:16 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:58:16.213254 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877130c0-828d-43ef-a6a5-c79c7002baf7 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:16.223347 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877130c0-828d-43ef-a6a5-c79c7002baf7 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:16.228645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d349e8-df8a-4155-894e-6561fc4f16c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:16.229242 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d349e8-df8a-4155-894e-6561fc4f16c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:16.229515 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d349e8-df8a-4155-894e-6561fc4f16c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:16.229782 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d349e8-df8a-4155-894e-6561fc4f16c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:16.230433 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d349e8-df8a-4155-894e-6561fc4f16c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:16.232046 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d349e8-df8a-4155-894e-6561fc4f16c1 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:16.241960 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1207/2434] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:58:16.245930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc780feb-76aa-41ec-a6e7-fcbfa94a4042 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:16.256139 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc780feb-76aa-41ec-a6e7-fcbfa94a4042 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:16.261491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea1f45-0494-4d65-8c0d-60c63ff910d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:16.262092 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea1f45-0494-4d65-8c0d-60c63ff910d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:16.262371 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea1f45-0494-4d65-8c0d-60c63ff910d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:16.262640 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea1f45-0494-4d65-8c0d-60c63ff910d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:16.263267 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea1f45-0494-4d65-8c0d-60c63ff910d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:16.264889 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea1f45-0494-4d65-8c0d-60c63ff910d4 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:16.276373 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1228/2435] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:58:16.280443 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d349e8-df8a-4155-894e-6561fc4f16c1 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:16.290893 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d349e8-df8a-4155-894e-6561fc4f16c1 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:16.296004 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426cf98-02e4-4e6e-bd62-e04fa8c93930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:16.296758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426cf98-02e4-4e6e-bd62-e04fa8c93930 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:16.297250 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426cf98-02e4-4e6e-bd62-e04fa8c93930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:16.297533 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426cf98-02e4-4e6e-bd62-e04fa8c93930 admin admin] PATH_INFO: `/v3/projects/dd698539bf9e48f5995aef97c2f875d9/users/f915ce7c2b1c4fb2bd53181542be3f42/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:16.312816 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426cf98-02e4-4e6e-bd62-e04fa8c93930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd698539bf9e48f5995aef97c2f875d9, user_id=f915ce7c2b1c4fb2bd53181542be3f42, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:16.315468 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426cf98-02e4-4e6e-bd62-e04fa8c93930 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:16.329055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1208/2436] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:58:16 2022] PUT /identity/v3/projects/dd698539bf9e48f5995aef97c2f875d9/users/f915ce7c2b1c4fb2bd53181542be3f42/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:58:16.340838 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b6db6-a69a-42d1-a8c3-bdd94e7bec71 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:16.341124 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b6db6-a69a-42d1-a8c3-bdd94e7bec71 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:16.341427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b6db6-a69a-42d1-a8c3-bdd94e7bec71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:16.370456 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-662b6db6-a69a-42d1-a8c3-bdd94e7bec71 None None] MFA Rules not processed for user `f915ce7c2b1c4fb2bd53181542be3f42`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:58:16.371696 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-662b6db6-a69a-42d1-a8c3-bdd94e7bec71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:16.437698 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1229/2437] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:58:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:58:16.441762 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426cf98-02e4-4e6e-bd62-e04fa8c93930 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:16.462519 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:16.504189 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426cf98-02e4-4e6e-bd62-e04fa8c93930 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:16.509157 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4d498-7111-407f-9a75-9b957cefde85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:16.509767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4d498-7111-407f-9a75-9b957cefde85 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:16.510046 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4d498-7111-407f-9a75-9b957cefde85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:16.510316 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4d498-7111-407f-9a75-9b957cefde85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:16.510942 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c4d498-7111-407f-9a75-9b957cefde85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:16.512872 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c4d498-7111-407f-9a75-9b957cefde85 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:16.545125 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1209/2438] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:58:16 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:58:16.549103 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b6db6-a69a-42d1-a8c3-bdd94e7bec71 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:16.559659 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b6db6-a69a-42d1-a8c3-bdd94e7bec71 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:16.572324 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:16.619190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8e62d-1018-4854-aec8-514023860058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:16.619759 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8e62d-1018-4854-aec8-514023860058 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:16.620035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8e62d-1018-4854-aec8-514023860058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:16.620300 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8e62d-1018-4854-aec8-514023860058 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:16.620914 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8e62d-1018-4854-aec8-514023860058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:16.622816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8e62d-1018-4854-aec8-514023860058 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:16.644546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1230/2439] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:58:16 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:58:16.649286 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4d498-7111-407f-9a75-9b957cefde85 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:16.659475 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4d498-7111-407f-9a75-9b957cefde85 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:16.666173 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59b6ed-30c0-4a0d-8676-a59d7374f0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:16.666835 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59b6ed-30c0-4a0d-8676-a59d7374f0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:16.667148 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59b6ed-30c0-4a0d-8676-a59d7374f0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:16.667389 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59b6ed-30c0-4a0d-8676-a59d7374f0a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:16.668106 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59b6ed-30c0-4a0d-8676-a59d7374f0a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:16.669684 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59b6ed-30c0-4a0d-8676-a59d7374f0a4 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:16.680219 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1210/2440] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:58:16.684129 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8e62d-1018-4854-aec8-514023860058 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:16.694549 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8e62d-1018-4854-aec8-514023860058 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:16.699155 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd57bc-7292-4b6c-87ce-5e65cd939610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:16.699780 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd57bc-7292-4b6c-87ce-5e65cd939610 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:16.700062 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd57bc-7292-4b6c-87ce-5e65cd939610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:16.700349 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd57bc-7292-4b6c-87ce-5e65cd939610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:16.700978 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd57bc-7292-4b6c-87ce-5e65cd939610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:16.702845 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd57bc-7292-4b6c-87ce-5e65cd939610 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:16.713722 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1231/2441] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:58:16.717890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59b6ed-30c0-4a0d-8676-a59d7374f0a4 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:16.727954 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59b6ed-30c0-4a0d-8676-a59d7374f0a4 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:16.732582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0f4af-9e7a-43e1-96d3-96bf1f111356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:16.733226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0f4af-9e7a-43e1-96d3-96bf1f111356 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:16.733500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0f4af-9e7a-43e1-96d3-96bf1f111356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:16.733772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0f4af-9e7a-43e1-96d3-96bf1f111356 admin admin] PATH_INFO: `/v3/projects/4e02560e929d42d7b91452768afde383/users/775bb56edc3742feb10ab42a72e7f2d8/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:16.748332 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0f4af-9e7a-43e1-96d3-96bf1f111356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e02560e929d42d7b91452768afde383, user_id=775bb56edc3742feb10ab42a72e7f2d8, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:16.750960 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0f4af-9e7a-43e1-96d3-96bf1f111356 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:16.765359 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1211/2442] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:58:16 2022] PUT /identity/v3/projects/4e02560e929d42d7b91452768afde383/users/775bb56edc3742feb10ab42a72e7f2d8/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:58:16.776095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd8bae-44c7-44bb-823f-e56c8f7f8518 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:16.776386 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd8bae-44c7-44bb-823f-e56c8f7f8518 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:16.776664 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd8bae-44c7-44bb-823f-e56c8f7f8518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:16.805818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-b5fd8bae-44c7-44bb-823f-e56c8f7f8518 None None] MFA Rules not processed for user `775bb56edc3742feb10ab42a72e7f2d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:58:16.807113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b5fd8bae-44c7-44bb-823f-e56c8f7f8518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:16.877484 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1232/2443] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:58:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:58:16.882476 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0f4af-9e7a-43e1-96d3-96bf1f111356 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:16.952981 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0f4af-9e7a-43e1-96d3-96bf1f111356 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:16.960170 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38564e-519d-4845-9cee-b322bbf4a17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:16.962499 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38564e-519d-4845-9cee-b322bbf4a17a admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:16.962784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38564e-519d-4845-9cee-b322bbf4a17a admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:16.964031 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38564e-519d-4845-9cee-b322bbf4a17a admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:16.965012 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38564e-519d-4845-9cee-b322bbf4a17a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:16.967436 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38564e-519d-4845-9cee-b322bbf4a17a admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:16.994238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1212/2444] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:58:16 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:58:16.998330 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd8bae-44c7-44bb-823f-e56c8f7f8518 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:17.010241 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd8bae-44c7-44bb-823f-e56c8f7f8518 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:17.065195 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec591f-e08b-4a8d-837e-2617e2b482ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:17.065796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec591f-e08b-4a8d-837e-2617e2b482ed admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:17.066089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec591f-e08b-4a8d-837e-2617e2b482ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:17.066379 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec591f-e08b-4a8d-837e-2617e2b482ed admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:17.066999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec591f-e08b-4a8d-837e-2617e2b482ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:17.069646 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec591f-e08b-4a8d-837e-2617e2b482ed admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:17.090048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1233/2445] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:58:16 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:58:17.094012 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38564e-519d-4845-9cee-b322bbf4a17a admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:17.104345 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38564e-519d-4845-9cee-b322bbf4a17a admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:17.108982 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a8a6b-a983-42ac-8052-3bec68a558f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:17.109576 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a8a6b-a983-42ac-8052-3bec68a558f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:17.109855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a8a6b-a983-42ac-8052-3bec68a558f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:17.110124 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a8a6b-a983-42ac-8052-3bec68a558f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:17.110754 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a8a6b-a983-42ac-8052-3bec68a558f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:17.112371 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a8a6b-a983-42ac-8052-3bec68a558f8 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:17.123235 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1213/2446] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:58:17.127254 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec591f-e08b-4a8d-837e-2617e2b482ed admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:17.137746 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec591f-e08b-4a8d-837e-2617e2b482ed admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:17.142359 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526ebe-ac91-4163-ad34-ed13872e30be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:17.143010 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526ebe-ac91-4163-ad34-ed13872e30be admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:17.143293 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526ebe-ac91-4163-ad34-ed13872e30be admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:17.143562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526ebe-ac91-4163-ad34-ed13872e30be admin admin] PATH_INFO: `/v3/projects/c1fdc9615fdf4ca285f9312e13a8402d/users/f8b333ef394945d98fb3c92dbd631fba/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:17.155519 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:17.159151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8526ebe-ac91-4163-ad34-ed13872e30be admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1fdc9615fdf4ca285f9312e13a8402d, user_id=f8b333ef394945d98fb3c92dbd631fba, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:17.162676 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8526ebe-ac91-4163-ad34-ed13872e30be admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:17.183491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1234/2447] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:58:17 2022] PUT /identity/v3/projects/c1fdc9615fdf4ca285f9312e13a8402d/users/f8b333ef394945d98fb3c92dbd631fba/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:58:17.187525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a8a6b-a983-42ac-8052-3bec68a558f8 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:17.198303 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a8a6b-a983-42ac-8052-3bec68a558f8 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:17.238911 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:17.255619 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36fb95-6fb2-4dae-b144-2266d00bd5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:17.256788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36fb95-6fb2-4dae-b144-2266d00bd5aa admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:17.257097 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36fb95-6fb2-4dae-b144-2266d00bd5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:17.257368 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36fb95-6fb2-4dae-b144-2266d00bd5aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:17.257996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36fb95-6fb2-4dae-b144-2266d00bd5aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:17.259866 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36fb95-6fb2-4dae-b144-2266d00bd5aa admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:17.269406 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1214/2448] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:58:17.273589 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526ebe-ac91-4163-ad34-ed13872e30be admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:17.284263 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526ebe-ac91-4163-ad34-ed13872e30be admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:17.289658 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d6e66-8329-4bac-a22f-90f8b2dfda33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:17.290316 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d6e66-8329-4bac-a22f-90f8b2dfda33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:17.290656 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d6e66-8329-4bac-a22f-90f8b2dfda33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:17.290854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d6e66-8329-4bac-a22f-90f8b2dfda33 admin admin] PATH_INFO: `/v3/projects/c1fdc9615fdf4ca285f9312e13a8402d/users/f8b333ef394945d98fb3c92dbd631fba/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:17.292779 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d6e66-8329-4bac-a22f-90f8b2dfda33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1fdc9615fdf4ca285f9312e13a8402d, user_id=f8b333ef394945d98fb3c92dbd631fba, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:17.295427 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d6e66-8329-4bac-a22f-90f8b2dfda33 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:17.306687 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1235/2449] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:58:17 2022] PUT /identity/v3/projects/c1fdc9615fdf4ca285f9312e13a8402d/users/f8b333ef394945d98fb3c92dbd631fba/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:58:17.316663 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfdd5e-0baf-495b-a176-fa26ff1f01ec None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:17.316951 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfdd5e-0baf-495b-a176-fa26ff1f01ec None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:17.317229 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfdd5e-0baf-495b-a176-fa26ff1f01ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:17.344205 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-c3cfdd5e-0baf-495b-a176-fa26ff1f01ec None None] MFA Rules not processed for user `f8b333ef394945d98fb3c92dbd631fba`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:58:17.345409 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-c3cfdd5e-0baf-495b-a176-fa26ff1f01ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:17.413255 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1215/2450] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:58:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:58:17.424261 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d6e66-8329-4bac-a22f-90f8b2dfda33 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:17.425953 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-465d6e66-8329-4bac-a22f-90f8b2dfda33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:17.481723 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d6e66-8329-4bac-a22f-90f8b2dfda33 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:17.485534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589778fd-5db9-42b9-911a-70e8d51fe345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:17.486155 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589778fd-5db9-42b9-911a-70e8d51fe345 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:17.486435 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589778fd-5db9-42b9-911a-70e8d51fe345 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:17.486730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589778fd-5db9-42b9-911a-70e8d51fe345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:17.543655 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589778fd-5db9-42b9-911a-70e8d51fe345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:17.545087 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589778fd-5db9-42b9-911a-70e8d51fe345 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:17.554241 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1236/2451] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:58:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:58:19.052138 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfdd5e-0baf-495b-a176-fa26ff1f01ec None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:19.062226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfdd5e-0baf-495b-a176-fa26ff1f01ec None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:19.065707 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a15954-a691-432e-8b5a-992be1336350 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:19.066296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a15954-a691-432e-8b5a-992be1336350 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:19.066573 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a15954-a691-432e-8b5a-992be1336350 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:19.066843 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a15954-a691-432e-8b5a-992be1336350 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:19.121042 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a15954-a691-432e-8b5a-992be1336350 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:19.122501 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a15954-a691-432e-8b5a-992be1336350 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:19.131490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1216/2452] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:58:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:58:19.176237 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589778fd-5db9-42b9-911a-70e8d51fe345 service cinder] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:19.178335 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-589778fd-5db9-42b9-911a-70e8d51fe345 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:19.235601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589778fd-5db9-42b9-911a-70e8d51fe345 service cinder] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:19.239660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03625f-e2e3-4548-a9c9-3ca373b2e1ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:19.240310 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03625f-e2e3-4548-a9c9-3ca373b2e1ed service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:19.240611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03625f-e2e3-4548-a9c9-3ca373b2e1ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:19.240852 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03625f-e2e3-4548-a9c9-3ca373b2e1ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:19.242934 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2a03625f-e2e3-4548-a9c9-3ca373b2e1ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:19.301687 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:19.329506 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03625f-e2e3-4548-a9c9-3ca373b2e1ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:19.331178 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03625f-e2e3-4548-a9c9-3ca373b2e1ed service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:19.344219 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1237/2453] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:58:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:58:19.676928 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a15954-a691-432e-8b5a-992be1336350 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:19.685609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a15954-a691-432e-8b5a-992be1336350 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:19.689289 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6874611-d2cf-47d3-a769-b47923dfe022 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:19.689892 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6874611-d2cf-47d3-a769-b47923dfe022 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:19.690169 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6874611-d2cf-47d3-a769-b47923dfe022 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:19.690440 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6874611-d2cf-47d3-a769-b47923dfe022 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:19.699851 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6874611-d2cf-47d3-a769-b47923dfe022 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:19.701253 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6874611-d2cf-47d3-a769-b47923dfe022 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:19.712986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1217/2454] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:58:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:58:20.525324 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03625f-e2e3-4548-a9c9-3ca373b2e1ed service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:20.526756 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2a03625f-e2e3-4548-a9c9-3ca373b2e1ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:20.537988 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03625f-e2e3-4548-a9c9-3ca373b2e1ed service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:20.541590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863e21d-d4da-49bb-b94e-8e0d4f1c3399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:20.542179 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863e21d-d4da-49bb-b94e-8e0d4f1c3399 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:20.542457 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863e21d-d4da-49bb-b94e-8e0d4f1c3399 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:20.542730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863e21d-d4da-49bb-b94e-8e0d4f1c3399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:20.544466 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9863e21d-d4da-49bb-b94e-8e0d4f1c3399 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:20.557981 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863e21d-d4da-49bb-b94e-8e0d4f1c3399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:20.559395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863e21d-d4da-49bb-b94e-8e0d4f1c3399 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:20.570329 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1238/2455] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:58:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:58:23.607067 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6874611-d2cf-47d3-a769-b47923dfe022 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:23.659848 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=100 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:23.672265 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6874611-d2cf-47d3-a769-b47923dfe022 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:23.676970 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88df2c-bdc1-4db4-a242-c526dee50130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:23.677629 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88df2c-bdc1-4db4-a242-c526dee50130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:23.677907 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88df2c-bdc1-4db4-a242-c526dee50130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:23.678178 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88df2c-bdc1-4db4-a242-c526dee50130 admin admin] PATH_INFO: `/v3/users/f915ce7c2b1c4fb2bd53181542be3f42` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:23.679323 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88df2c-bdc1-4db4-a242-c526dee50130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f915ce7c2b1c4fb2bd53181542be3f42)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:23.681277 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88df2c-bdc1-4db4-a242-c526dee50130 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:23.729698 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ec88df2c-bdc1-4db4-a242-c526dee50130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f915ce7c2b1c4fb2bd53181542be3f42'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:23.730351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ec88df2c-bdc1-4db4-a242-c526dee50130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f915ce7c2b1c4fb2bd53181542be3f42'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:23.746446 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ec88df2c-bdc1-4db4-a242-c526dee50130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f915ce7c2b1c4fb2bd53181542be3f42'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:23.754510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ec88df2c-bdc1-4db4-a242-c526dee50130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f915ce7c2b1c4fb2bd53181542be3f42'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:23.755227 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-ec88df2c-bdc1-4db4-a242-c526dee50130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f915ce7c2b1c4fb2bd53181542be3f42'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:23.767890 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1218/2456] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:58:23 2022] DELETE /identity/v3/users/f915ce7c2b1c4fb2bd53181542be3f42 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:58:23.773012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5524e-1f57-4390-af8a-1a23bae4ac1b None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:23.773494 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5524e-1f57-4390-af8a-1a23bae4ac1b None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:23.773725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5524e-1f57-4390-af8a-1a23bae4ac1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:23.793192 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-57e5524e-1f57-4390-af8a-1a23bae4ac1b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:58:23.794489 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-57e5524e-1f57-4390-af8a-1a23bae4ac1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:23.855155 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1239/2457] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:58:23.865273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88df2c-bdc1-4db4-a242-c526dee50130 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:23.866726 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-ec88df2c-bdc1-4db4-a242-c526dee50130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:23.933470 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88df2c-bdc1-4db4-a242-c526dee50130 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:23.938241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122dc06-1849-4981-9e35-1e6c511b7737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:23.938800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122dc06-1849-4981-9e35-1e6c511b7737 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:23.939720 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122dc06-1849-4981-9e35-1e6c511b7737 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:23.939999 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122dc06-1849-4981-9e35-1e6c511b7737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:23.959234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7122dc06-1849-4981-9e35-1e6c511b7737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:23.961569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7122dc06-1849-4981-9e35-1e6c511b7737 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:23.974385 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1219/2458] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:58:24.313899 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5524e-1f57-4390-af8a-1a23bae4ac1b None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:24.315376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-57e5524e-1f57-4390-af8a-1a23bae4ac1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:24.329414 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5524e-1f57-4390-af8a-1a23bae4ac1b None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:24.333994 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f6736-30fb-4683-98bb-0a459e5f393a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:24.335197 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f6736-30fb-4683-98bb-0a459e5f393a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:24.336018 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f6736-30fb-4683-98bb-0a459e5f393a admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:24.336525 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f6736-30fb-4683-98bb-0a459e5f393a admin admin] PATH_INFO: `/v3/projects/dd698539bf9e48f5995aef97c2f875d9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:24.337734 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f6736-30fb-4683-98bb-0a459e5f393a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd698539bf9e48f5995aef97c2f875d9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:24.339559 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f6736-30fb-4683-98bb-0a459e5f393a admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:24.385390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ba1f6736-30fb-4683-98bb-0a459e5f393a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd698539bf9e48f5995aef97c2f875d9', 'request_id': 'req-ba1f6736-30fb-4683-98bb-0a459e5f393a'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:24.391517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ba1f6736-30fb-4683-98bb-0a459e5f393a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd698539bf9e48f5995aef97c2f875d9', 'request_id': 'req-ba1f6736-30fb-4683-98bb-0a459e5f393a'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:24.399982 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ba1f6736-30fb-4683-98bb-0a459e5f393a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd698539bf9e48f5995aef97c2f875d9', 'request_id': 'req-ba1f6736-30fb-4683-98bb-0a459e5f393a'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:24.410063 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ba1f6736-30fb-4683-98bb-0a459e5f393a admin admin] The token cache is being invalidate because project dd698539bf9e48f5995aef97c2f875d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:58:24.410347 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ba1f6736-30fb-4683-98bb-0a459e5f393a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:24.410964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-ba1f6736-30fb-4683-98bb-0a459e5f393a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:24.412475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1240/2459] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:58:24 2022] DELETE /identity/v3/projects/dd698539bf9e48f5995aef97c2f875d9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:58:24.416459 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122dc06-1849-4981-9e35-1e6c511b7737 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:24.426077 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122dc06-1849-4981-9e35-1e6c511b7737 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:24.482196 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d33f3c-d4fe-4ac5-b7e6-09df607783fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:24.482899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d33f3c-d4fe-4ac5-b7e6-09df607783fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:24.483245 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d33f3c-d4fe-4ac5-b7e6-09df607783fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:24.483638 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d33f3c-d4fe-4ac5-b7e6-09df607783fe admin admin] PATH_INFO: `/v3/users/775bb56edc3742feb10ab42a72e7f2d8` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:24.484831 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d33f3c-d4fe-4ac5-b7e6-09df607783fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=775bb56edc3742feb10ab42a72e7f2d8)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:24.486797 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d33f3c-d4fe-4ac5-b7e6-09df607783fe admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:24.501816 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:24.537510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-84d33f3c-d4fe-4ac5-b7e6-09df607783fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '775bb56edc3742feb10ab42a72e7f2d8'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:24.538352 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-84d33f3c-d4fe-4ac5-b7e6-09df607783fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '775bb56edc3742feb10ab42a72e7f2d8'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:24.550513 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-84d33f3c-d4fe-4ac5-b7e6-09df607783fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '775bb56edc3742feb10ab42a72e7f2d8'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:24.558315 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-84d33f3c-d4fe-4ac5-b7e6-09df607783fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '775bb56edc3742feb10ab42a72e7f2d8'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:24.559439 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-84d33f3c-d4fe-4ac5-b7e6-09df607783fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '775bb56edc3742feb10ab42a72e7f2d8'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:24.574926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1220/2460] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:58:24 2022] DELETE /identity/v3/users/775bb56edc3742feb10ab42a72e7f2d8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:58:24.844750 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f6736-30fb-4683-98bb-0a459e5f393a admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:24.846489 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ba1f6736-30fb-4683-98bb-0a459e5f393a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:24.863608 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f6736-30fb-4683-98bb-0a459e5f393a admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:24.910872 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:24.927710 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5604a6f-9c91-49be-9681-ab54f18a28ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:24.928960 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5604a6f-9c91-49be-9681-ab54f18a28ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:24.929244 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5604a6f-9c91-49be-9681-ab54f18a28ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:24.929516 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5604a6f-9c91-49be-9681-ab54f18a28ab admin admin] PATH_INFO: `/v3/projects/4e02560e929d42d7b91452768afde383` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:24.930621 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5604a6f-9c91-49be-9681-ab54f18a28ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e02560e929d42d7b91452768afde383)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:24.932576 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5604a6f-9c91-49be-9681-ab54f18a28ab admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:24.980019 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e5604a6f-9c91-49be-9681-ab54f18a28ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e02560e929d42d7b91452768afde383', 'request_id': 'req-e5604a6f-9c91-49be-9681-ab54f18a28ab'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:24.986256 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e5604a6f-9c91-49be-9681-ab54f18a28ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e02560e929d42d7b91452768afde383', 'request_id': 'req-e5604a6f-9c91-49be-9681-ab54f18a28ab'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:24.994480 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e5604a6f-9c91-49be-9681-ab54f18a28ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e02560e929d42d7b91452768afde383', 'request_id': 'req-e5604a6f-9c91-49be-9681-ab54f18a28ab'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:25.005421 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e5604a6f-9c91-49be-9681-ab54f18a28ab admin admin] The token cache is being invalidate because project 4e02560e929d42d7b91452768afde383 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:58:25.005734 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e5604a6f-9c91-49be-9681-ab54f18a28ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:25.006383 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e5604a6f-9c91-49be-9681-ab54f18a28ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:25.008133 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1241/2461] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:58:24 2022] DELETE /identity/v3/projects/4e02560e929d42d7b91452768afde383 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:58:25.012284 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d33f3c-d4fe-4ac5-b7e6-09df607783fe admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:25.013682 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-84d33f3c-d4fe-4ac5-b7e6-09df607783fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:25.026623 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d33f3c-d4fe-4ac5-b7e6-09df607783fe admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:25.084104 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d1e73-f551-4637-8dd3-6a9ca976201d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:25.084770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d1e73-f551-4637-8dd3-6a9ca976201d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:25.085045 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d1e73-f551-4637-8dd3-6a9ca976201d admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:25.085316 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d1e73-f551-4637-8dd3-6a9ca976201d admin admin] PATH_INFO: `/v3/users/f8b333ef394945d98fb3c92dbd631fba` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:25.086456 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d1e73-f551-4637-8dd3-6a9ca976201d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8b333ef394945d98fb3c92dbd631fba)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:25.088435 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d1e73-f551-4637-8dd3-6a9ca976201d admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:25.153642 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-165d1e73-f551-4637-8dd3-6a9ca976201d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8b333ef394945d98fb3c92dbd631fba'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:25.154298 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-165d1e73-f551-4637-8dd3-6a9ca976201d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8b333ef394945d98fb3c92dbd631fba'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:25.167417 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-165d1e73-f551-4637-8dd3-6a9ca976201d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8b333ef394945d98fb3c92dbd631fba'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:25.175756 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-165d1e73-f551-4637-8dd3-6a9ca976201d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8b333ef394945d98fb3c92dbd631fba'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:25.176372 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-165d1e73-f551-4637-8dd3-6a9ca976201d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8b333ef394945d98fb3c92dbd631fba'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:25.194146 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1221/2462] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:58:25 2022] DELETE /identity/v3/users/f8b333ef394945d98fb3c92dbd631fba => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:58:25.527651 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5604a6f-9c91-49be-9681-ab54f18a28ab admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:25.529198 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e5604a6f-9c91-49be-9681-ab54f18a28ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:25.542354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5604a6f-9c91-49be-9681-ab54f18a28ab admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:25.616484 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a355a13-c9e1-4eeb-9097-0e0a9241667f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:25.617460 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a355a13-c9e1-4eeb-9097-0e0a9241667f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:25.617913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a355a13-c9e1-4eeb-9097-0e0a9241667f admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:25.618391 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a355a13-c9e1-4eeb-9097-0e0a9241667f admin admin] PATH_INFO: `/v3/projects/c1fdc9615fdf4ca285f9312e13a8402d` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:25.619838 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a355a13-c9e1-4eeb-9097-0e0a9241667f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1fdc9615fdf4ca285f9312e13a8402d)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:25.622531 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a355a13-c9e1-4eeb-9097-0e0a9241667f admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:25.689092 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0a355a13-c9e1-4eeb-9097-0e0a9241667f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1fdc9615fdf4ca285f9312e13a8402d', 'request_id': 'req-0a355a13-c9e1-4eeb-9097-0e0a9241667f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:25.696677 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0a355a13-c9e1-4eeb-9097-0e0a9241667f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1fdc9615fdf4ca285f9312e13a8402d', 'request_id': 'req-0a355a13-c9e1-4eeb-9097-0e0a9241667f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:25.704243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0a355a13-c9e1-4eeb-9097-0e0a9241667f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1fdc9615fdf4ca285f9312e13a8402d', 'request_id': 'req-0a355a13-c9e1-4eeb-9097-0e0a9241667f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:25.716560 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0a355a13-c9e1-4eeb-9097-0e0a9241667f admin admin] The token cache is being invalidate because project c1fdc9615fdf4ca285f9312e13a8402d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:58:25.716847 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0a355a13-c9e1-4eeb-9097-0e0a9241667f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:25.717482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0a355a13-c9e1-4eeb-9097-0e0a9241667f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:25.718969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1242/2463] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:58:25 2022] DELETE /identity/v3/projects/c1fdc9615fdf4ca285f9312e13a8402d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:58:25.724694 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47724ccc-7448-4141-a68f-509fe8c732c8 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:25.725000 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47724ccc-7448-4141-a68f-509fe8c732c8 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:25.725811 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47724ccc-7448-4141-a68f-509fe8c732c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:25.746664 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-47724ccc-7448-4141-a68f-509fe8c732c8 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:58:25.747879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-47724ccc-7448-4141-a68f-509fe8c732c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:25.753182 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:25.811687 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1222/2464] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:58:25.819532 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9412eac-8cd9-411c-8dc4-9d4588085f53 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:25.819844 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9412eac-8cd9-411c-8dc4-9d4588085f53 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:25.820600 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9412eac-8cd9-411c-8dc4-9d4588085f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:25.829834 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:25.842368 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-a9412eac-8cd9-411c-8dc4-9d4588085f53 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:58:25.844096 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-a9412eac-8cd9-411c-8dc4-9d4588085f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:25.852622 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1243/2465] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:58:25.856711 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47724ccc-7448-4141-a68f-509fe8c732c8 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:25.866921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47724ccc-7448-4141-a68f-509fe8c732c8 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:25.871367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d1445-bf2d-49ee-a1b1-cc7830da90f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:25.871999 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d1445-bf2d-49ee-a1b1-cc7830da90f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:25.872362 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d1445-bf2d-49ee-a1b1-cc7830da90f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:25.872611 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d1445-bf2d-49ee-a1b1-cc7830da90f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:25.873247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d1445-bf2d-49ee-a1b1-cc7830da90f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 21:58:25.873544 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d1445-bf2d-49ee-a1b1-cc7830da90f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:25.875119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d1445-bf2d-49ee-a1b1-cc7830da90f9 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:25.886978 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1223/2466] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 21:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 21:58:25.891752 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9412eac-8cd9-411c-8dc4-9d4588085f53 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:25.901561 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9412eac-8cd9-411c-8dc4-9d4588085f53 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:25.906193 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d2bf5-88af-45b8-96c8-608dfa60c8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:25.906793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d2bf5-88af-45b8-96c8-608dfa60c8d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:25.907069 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d2bf5-88af-45b8-96c8-608dfa60c8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:25.907338 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d2bf5-88af-45b8-96c8-608dfa60c8d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:25.907999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d2bf5-88af-45b8-96c8-608dfa60c8d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:25.910384 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d2bf5-88af-45b8-96c8-608dfa60c8d0 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:25.941836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1244/2467] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:58:25 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:58:25.943152 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d1445-bf2d-49ee-a1b1-cc7830da90f9 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:25.953142 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d1445-bf2d-49ee-a1b1-cc7830da90f9 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:26.007334 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbb9c0-0725-4bed-bcdf-fb66d4d79bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:26.007955 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbb9c0-0725-4bed-bcdf-fb66d4d79bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:26.008231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbb9c0-0725-4bed-bcdf-fb66d4d79bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:26.008500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbb9c0-0725-4bed-bcdf-fb66d4d79bc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:26.009477 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dbb9c0-0725-4bed-bcdf-fb66d4d79bc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:26.011370 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dbb9c0-0725-4bed-bcdf-fb66d4d79bc2 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:26.032557 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1224/2468] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:58:25 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:58:26.036701 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d2bf5-88af-45b8-96c8-608dfa60c8d0 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:26.047542 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d2bf5-88af-45b8-96c8-608dfa60c8d0 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:26.052208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254f1bd-0faf-4ad3-9a95-2b29d84ebbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:26.052797 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254f1bd-0faf-4ad3-9a95-2b29d84ebbf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:26.053075 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254f1bd-0faf-4ad3-9a95-2b29d84ebbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:26.053344 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254f1bd-0faf-4ad3-9a95-2b29d84ebbf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:26.053969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254f1bd-0faf-4ad3-9a95-2b29d84ebbf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:26.055648 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254f1bd-0faf-4ad3-9a95-2b29d84ebbf1 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:26.065517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1245/2469] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:58:26.069232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbb9c0-0725-4bed-bcdf-fb66d4d79bc2 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:26.079052 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbb9c0-0725-4bed-bcdf-fb66d4d79bc2 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:26.083517 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0812a-87ac-4383-a472-7a38199ff3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:26.084137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0812a-87ac-4383-a472-7a38199ff3fc admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:26.084414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0812a-87ac-4383-a472-7a38199ff3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:26.085369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0812a-87ac-4383-a472-7a38199ff3fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:26.086001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0812a-87ac-4383-a472-7a38199ff3fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:26.087604 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0812a-87ac-4383-a472-7a38199ff3fc admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:26.097950 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1225/2470] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:58:26.102379 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254f1bd-0faf-4ad3-9a95-2b29d84ebbf1 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:26.112165 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254f1bd-0faf-4ad3-9a95-2b29d84ebbf1 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:26.116682 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b3e4b-a558-485f-9f4b-8dc7edbbea6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:26.117318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b3e4b-a558-485f-9f4b-8dc7edbbea6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:26.117597 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b3e4b-a558-485f-9f4b-8dc7edbbea6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:26.117871 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b3e4b-a558-485f-9f4b-8dc7edbbea6f admin admin] PATH_INFO: `/v3/projects/15649df008a249a39e6d71c40a020954/users/05ec7e8b5ae04b2d84323ececfbf5acc/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:26.132954 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b3e4b-a558-485f-9f4b-8dc7edbbea6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=15649df008a249a39e6d71c40a020954, user_id=05ec7e8b5ae04b2d84323ececfbf5acc, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:26.135906 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b3e4b-a558-485f-9f4b-8dc7edbbea6f admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:26.148917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1246/2471] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:58:26 2022] PUT /identity/v3/projects/15649df008a249a39e6d71c40a020954/users/05ec7e8b5ae04b2d84323ececfbf5acc/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 21:58:26.153850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd3213-f067-471f-8163-a6ca80764283 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:26.154126 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd3213-f067-471f-8163-a6ca80764283 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:26.154390 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd3213-f067-471f-8163-a6ca80764283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:26.173211 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-95fd3213-f067-471f-8163-a6ca80764283 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:58:26.174420 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-95fd3213-f067-471f-8163-a6ca80764283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:26.233464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1226/2472] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:58:26.243078 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b3e4b-a558-485f-9f4b-8dc7edbbea6f admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:26.244782 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-682b3e4b-a558-485f-9f4b-8dc7edbbea6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:26.300377 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b3e4b-a558-485f-9f4b-8dc7edbbea6f admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:26.304018 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e6c90-2cd1-469a-9e90-ab223463dc25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:26.304610 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e6c90-2cd1-469a-9e90-ab223463dc25 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:26.304881 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e6c90-2cd1-469a-9e90-ab223463dc25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:26.305147 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e6c90-2cd1-469a-9e90-ab223463dc25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:26.313778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e6c90-2cd1-469a-9e90-ab223463dc25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:26.315197 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e6c90-2cd1-469a-9e90-ab223463dc25 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:26.325078 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1247/2473] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:58:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 21:58:30.455277 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da81a95-7833-4313-a2e4-acdd02c38377 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:30.455806 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da81a95-7833-4313-a2e4-acdd02c38377 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:30.455897 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da81a95-7833-4313-a2e4-acdd02c38377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:30.498880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-1da81a95-7833-4313-a2e4-acdd02c38377 None None] MFA Rules not processed for user `05ec7e8b5ae04b2d84323ececfbf5acc`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:58:30.500253 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1da81a95-7833-4313-a2e4-acdd02c38377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:30.505688 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:30.571871 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1227/2474] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:58:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:58:30.576477 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e6c90-2cd1-469a-9e90-ab223463dc25 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:30.587029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e6c90-2cd1-469a-9e90-ab223463dc25 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:30.591731 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af6efb-d335-4b35-926b-daf80fbabc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:30.592428 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af6efb-d335-4b35-926b-daf80fbabc1d admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:30.592707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af6efb-d335-4b35-926b-daf80fbabc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:30.592979 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af6efb-d335-4b35-926b-daf80fbabc1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:30.593615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af6efb-d335-4b35-926b-daf80fbabc1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:30.595546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af6efb-d335-4b35-926b-daf80fbabc1d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:30.624087 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1248/2475] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 21:58:30 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 21:58:30.628279 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da81a95-7833-4313-a2e4-acdd02c38377 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:30.639855 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da81a95-7833-4313-a2e4-acdd02c38377 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:30.698601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a368c7-c1c5-4468-96ea-778404be122d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:30.699158 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a368c7-c1c5-4468-96ea-778404be122d admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:30.699485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a368c7-c1c5-4468-96ea-778404be122d admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:30.699763 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a368c7-c1c5-4468-96ea-778404be122d admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:30.700398 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a368c7-c1c5-4468-96ea-778404be122d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:30.702413 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a368c7-c1c5-4468-96ea-778404be122d admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:30.725598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1228/2476] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 21:58:30 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 21:58:30.729683 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af6efb-d335-4b35-926b-daf80fbabc1d admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:30.741118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af6efb-d335-4b35-926b-daf80fbabc1d admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:30.745783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d76fe0-e56d-47f5-97d5-2813c9bdcb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:30.746390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d76fe0-e56d-47f5-97d5-2813c9bdcb9b admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:30.746669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d76fe0-e56d-47f5-97d5-2813c9bdcb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:30.746939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d76fe0-e56d-47f5-97d5-2813c9bdcb9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:30.747584 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d76fe0-e56d-47f5-97d5-2813c9bdcb9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:30.749212 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d76fe0-e56d-47f5-97d5-2813c9bdcb9b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:30.761106 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1249/2477] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 21:58:30.765514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a368c7-c1c5-4468-96ea-778404be122d admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:30.776309 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a368c7-c1c5-4468-96ea-778404be122d admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:30.780925 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95db1aa-f8b5-460b-b19a-79fa2ce518a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:30.781583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95db1aa-f8b5-460b-b19a-79fa2ce518a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:30.781864 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95db1aa-f8b5-460b-b19a-79fa2ce518a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:30.782139 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95db1aa-f8b5-460b-b19a-79fa2ce518a3 admin admin] PATH_INFO: `/v3/projects/cf30f933d49d427495042cc70ef2ac7a/users/9fd03c04e4194579be1822ba5f0d2320/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:30.797487 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95db1aa-f8b5-460b-b19a-79fa2ce518a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf30f933d49d427495042cc70ef2ac7a, user_id=9fd03c04e4194579be1822ba5f0d2320, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:30.800201 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95db1aa-f8b5-460b-b19a-79fa2ce518a3 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:30.813716 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1229/2478] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:58:30 2022] PUT /identity/v3/projects/cf30f933d49d427495042cc70ef2ac7a/users/9fd03c04e4194579be1822ba5f0d2320/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:58:30.817850 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d76fe0-e56d-47f5-97d5-2813c9bdcb9b admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:30.827948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d76fe0-e56d-47f5-97d5-2813c9bdcb9b admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:30.881575 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325090a-59e8-4ac8-b57c-d9ea238a1dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:30.882168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325090a-59e8-4ac8-b57c-d9ea238a1dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:30.882445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325090a-59e8-4ac8-b57c-d9ea238a1dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:30.882715 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325090a-59e8-4ac8-b57c-d9ea238a1dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:30.883335 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325090a-59e8-4ac8-b57c-d9ea238a1dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:30.885200 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325090a-59e8-4ac8-b57c-d9ea238a1dd9 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:30.894398 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1250/2479] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 21:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 21:58:30.898621 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95db1aa-f8b5-460b-b19a-79fa2ce518a3 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:30.908675 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95db1aa-f8b5-460b-b19a-79fa2ce518a3 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:30.913826 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d3369-2d0c-4c4f-aeff-cfc324b075bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:30.914815 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d3369-2d0c-4c4f-aeff-cfc324b075bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:30.914815 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d3369-2d0c-4c4f-aeff-cfc324b075bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:30.914970 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d3369-2d0c-4c4f-aeff-cfc324b075bc admin admin] PATH_INFO: `/v3/projects/cf30f933d49d427495042cc70ef2ac7a/users/9fd03c04e4194579be1822ba5f0d2320/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:30.916911 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d3369-2d0c-4c4f-aeff-cfc324b075bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf30f933d49d427495042cc70ef2ac7a, user_id=9fd03c04e4194579be1822ba5f0d2320, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:30.919598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d3369-2d0c-4c4f-aeff-cfc324b075bc admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:30.930757 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1230/2480] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 21:58:30 2022] PUT /identity/v3/projects/cf30f933d49d427495042cc70ef2ac7a/users/9fd03c04e4194579be1822ba5f0d2320/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:58:31.992818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf981a-d89a-4b4c-939f-ac5e3c596405 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:31.993671 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf981a-d89a-4b4c-939f-ac5e3c596405 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:31.994126 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf981a-d89a-4b4c-939f-ac5e3c596405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:32.013846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:32.020633 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-47cf981a-d89a-4b4c-939f-ac5e3c596405 None None] MFA Rules not processed for user `2346003afe74477496045e9bc72173d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:58:32.022612 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-47cf981a-d89a-4b4c-939f-ac5e3c596405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:32.093622 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1251/2481] 10.222.0.13 () {58 vars in 1009 bytes} [Fri Oct 21 21:58:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:58:32.145709 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d3369-2d0c-4c4f-aeff-cfc324b075bc admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:32.152205 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-f94d3369-2d0c-4c4f-aeff-cfc324b075bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:32.254570 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:32.263643 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d3369-2d0c-4c4f-aeff-cfc324b075bc admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:32.270289 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d229a8-4dd8-475c-a704-20b3795513b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ce720c8df014f9a8af062138aecd1b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:32.271559 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d229a8-4dd8-475c-a704-20b3795513b8 service swift] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:32.271954 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d229a8-4dd8-475c-a704-20b3795513b8 service swift] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:32.272534 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d229a8-4dd8-475c-a704-20b3795513b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:32.292697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d229a8-4dd8-475c-a704-20b3795513b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:32.294926 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d229a8-4dd8-475c-a704-20b3795513b8 service swift] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:32.306648 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1231/2482] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Oct 21 21:58:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 21:58:36.712133 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bb6e8-f565-47e8-b2e4-8bda4d0e8a2e None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:36.712785 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bb6e8-f565-47e8-b2e4-8bda4d0e8a2e None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:36.712888 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bb6e8-f565-47e8-b2e4-8bda4d0e8a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:36.771430 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-578bb6e8-f565-47e8-b2e4-8bda4d0e8a2e None None] MFA Rules not processed for user `9fd03c04e4194579be1822ba5f0d2320`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:58:36.773556 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-578bb6e8-f565-47e8-b2e4-8bda4d0e8a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:36.904110 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1252/2483] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:58:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:58:36.919010 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d229a8-4dd8-475c-a704-20b3795513b8 service swift] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:36.920647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-37d229a8-4dd8-475c-a704-20b3795513b8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:37.025587 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d229a8-4dd8-475c-a704-20b3795513b8 service swift] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:37.033694 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322b587-8862-4dab-ae11-66bc22660cee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:37.034472 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322b587-8862-4dab-ae11-66bc22660cee service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:37.034873 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322b587-8862-4dab-ae11-66bc22660cee service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:37.035359 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322b587-8862-4dab-ae11-66bc22660cee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:37.108734 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322b587-8862-4dab-ae11-66bc22660cee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:37.110580 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322b587-8862-4dab-ae11-66bc22660cee service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:37.123415 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1232/2484] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:58:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:58:39.469730 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3c294-fe75-4454-a9ec-0f675472b1f1 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:39.470281 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3c294-fe75-4454-a9ec-0f675472b1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:39.470281 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3c294-fe75-4454-a9ec-0f675472b1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:39.496569 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-91d3c294-fe75-4454-a9ec-0f675472b1f1 None None] MFA Rules not processed for user `05ec7e8b5ae04b2d84323ececfbf5acc`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:58:39.498443 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-91d3c294-fe75-4454-a9ec-0f675472b1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:39.509093 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1253/2485] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 21:58:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:58:47.116945 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=61 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:49.488596 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=59 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:58:54.476416 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d3386-82ba-43ce-9c7e-3821452ce324 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:54.476917 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d3386-82ba-43ce-9c7e-3821452ce324 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:54.477007 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d3386-82ba-43ce-9c7e-3821452ce324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:54.495582 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-216d3386-82ba-43ce-9c7e-3821452ce324 None None] MFA Rules not processed for user `2346003afe74477496045e9bc72173d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 21:58:54.496774 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-216d3386-82ba-43ce-9c7e-3821452ce324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:54.503766 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1233/2486] 10.222.0.13 () {58 vars in 967 bytes} [Fri Oct 21 21:58:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 21:58:54.516067 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d3c294-fe75-4454-a9ec-0f675472b1f1 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:54.528176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d3c294-fe75-4454-a9ec-0f675472b1f1 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:54.533270 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab038d39-ebf3-4f36-ad3d-10790fee3430 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ce720c8df014f9a8af062138aecd1b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:54.533869 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab038d39-ebf3-4f36-ad3d-10790fee3430 service swift] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:54.534150 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab038d39-ebf3-4f36-ad3d-10790fee3430 service swift] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:54.534421 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab038d39-ebf3-4f36-ad3d-10790fee3430 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:54.542891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab038d39-ebf3-4f36-ad3d-10790fee3430 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:54.544500 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab038d39-ebf3-4f36-ad3d-10790fee3430 service swift] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:54.552196 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1254/2487] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Oct 21 21:58:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 21:58:56.386793 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d3386-82ba-43ce-9c7e-3821452ce324 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:56.388220 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-216d3386-82ba-43ce-9c7e-3821452ce324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:56.437680 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab038d39-ebf3-4f36-ad3d-10790fee3430 service swift] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:56.439595 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ab038d39-ebf3-4f36-ad3d-10790fee3430 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:56.467462 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d3386-82ba-43ce-9c7e-3821452ce324 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:56.472474 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ff301-6d4b-419f-86a0-3bc0c26f28f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:56.473111 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ff301-6d4b-419f-86a0-3bc0c26f28f4 service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:56.473391 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ff301-6d4b-419f-86a0-3bc0c26f28f4 service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:56.473682 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ff301-6d4b-419f-86a0-3bc0c26f28f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:56.475430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-145ff301-6d4b-419f-86a0-3bc0c26f28f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:58:56.487498 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ff301-6d4b-419f-86a0-3bc0c26f28f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:56.489314 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ff301-6d4b-419f-86a0-3bc0c26f28f4 service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:56.499272 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1234/2488] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:58:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:58:56.507601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab038d39-ebf3-4f36-ad3d-10790fee3430 service swift] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:56.516546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff816db-2db0-4775-b9b0-373e189c6b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:56.517184 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff816db-2db0-4775-b9b0-373e189c6b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:56.517463 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff816db-2db0-4775-b9b0-373e189c6b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:56.517739 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff816db-2db0-4775-b9b0-373e189c6b8e admin admin] PATH_INFO: `/v3/users/a665b883f5ab4c408329c358da8972cf` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:56.519022 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff816db-2db0-4775-b9b0-373e189c6b8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a665b883f5ab4c408329c358da8972cf)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:56.521399 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff816db-2db0-4775-b9b0-373e189c6b8e admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:56.566451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7ff816db-2db0-4775-b9b0-373e189c6b8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a665b883f5ab4c408329c358da8972cf'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:56.578319 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7ff816db-2db0-4775-b9b0-373e189c6b8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a665b883f5ab4c408329c358da8972cf'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:56.585765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7ff816db-2db0-4775-b9b0-373e189c6b8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a665b883f5ab4c408329c358da8972cf'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:56.586502 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7ff816db-2db0-4775-b9b0-373e189c6b8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a665b883f5ab4c408329c358da8972cf'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:56.593320 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7ff816db-2db0-4775-b9b0-373e189c6b8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a665b883f5ab4c408329c358da8972cf'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:56.599046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1255/2489] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 21:58:56 2022] DELETE /identity/v3/users/a665b883f5ab4c408329c358da8972cf => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:58:56.771239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ff301-6d4b-419f-86a0-3bc0c26f28f4 service placement] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:58:56.782832 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ff301-6d4b-419f-86a0-3bc0c26f28f4 service placement] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:58:56.840025 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9083786-9f0c-4401-be01-e2f553076d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:58:56.840586 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9083786-9f0c-4401-be01-e2f553076d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:58:56.840875 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9083786-9f0c-4401-be01-e2f553076d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:58:56.841137 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9083786-9f0c-4401-be01-e2f553076d2c admin admin] PATH_INFO: `/v3/projects/bdafb20ebf674ea6a7f82876c7724341` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:58:56.842232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9083786-9f0c-4401-be01-e2f553076d2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdafb20ebf674ea6a7f82876c7724341)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:58:56.844154 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9083786-9f0c-4401-be01-e2f553076d2c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:58:56.892921 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f9083786-9f0c-4401-be01-e2f553076d2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdafb20ebf674ea6a7f82876c7724341', 'request_id': 'req-f9083786-9f0c-4401-be01-e2f553076d2c'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:56.899986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f9083786-9f0c-4401-be01-e2f553076d2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdafb20ebf674ea6a7f82876c7724341', 'request_id': 'req-f9083786-9f0c-4401-be01-e2f553076d2c'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:56.905583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f9083786-9f0c-4401-be01-e2f553076d2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdafb20ebf674ea6a7f82876c7724341', 'request_id': 'req-f9083786-9f0c-4401-be01-e2f553076d2c'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:56.916392 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f9083786-9f0c-4401-be01-e2f553076d2c admin admin] The token cache is being invalidate because project bdafb20ebf674ea6a7f82876c7724341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 21:58:56.916676 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f9083786-9f0c-4401-be01-e2f553076d2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:56.917288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-f9083786-9f0c-4401-be01-e2f553076d2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 21:58:56.918765 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1235/2490] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 21:58:56 2022] DELETE /identity/v3/projects/bdafb20ebf674ea6a7f82876c7724341 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 21:59:06.595654 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=59,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:59:06.913473 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:59:08.026187 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff816db-2db0-4775-b9b0-373e189c6b8e admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:59:08.027645 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-7ff816db-2db0-4775-b9b0-373e189c6b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:59:08.081770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff816db-2db0-4775-b9b0-373e189c6b8e admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:59:08.128048 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc30bb-733a-4a67-bfdf-a0f699aa99e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:59:08.128642 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc30bb-733a-4a67-bfdf-a0f699aa99e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:59:08.128918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc30bb-733a-4a67-bfdf-a0f699aa99e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:59:08.129187 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc30bb-733a-4a67-bfdf-a0f699aa99e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:59:08.131372 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f4bc30bb-733a-4a67-bfdf-a0f699aa99e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:59:08.140483 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc30bb-733a-4a67-bfdf-a0f699aa99e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:59:08.141895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc30bb-733a-4a67-bfdf-a0f699aa99e7 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:59:08.150468 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1256/2491] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:59:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:59:08.603485 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9083786-9f0c-4401-be01-e2f553076d2c admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:59:08.604957 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-f9083786-9f0c-4401-be01-e2f553076d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:59:08.681063 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9083786-9f0c-4401-be01-e2f553076d2c admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:59:08.685904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debb2f3-aa61-426d-b0cf-e7acb90fae9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:59:08.686527 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debb2f3-aa61-426d-b0cf-e7acb90fae9e service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:59:08.686800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debb2f3-aa61-426d-b0cf-e7acb90fae9e service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:59:08.687072 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debb2f3-aa61-426d-b0cf-e7acb90fae9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:59:08.688769 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-6debb2f3-aa61-426d-b0cf-e7acb90fae9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:59:08.700412 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debb2f3-aa61-426d-b0cf-e7acb90fae9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:59:08.701961 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debb2f3-aa61-426d-b0cf-e7acb90fae9e service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:59:08.712110 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1236/2492] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:59:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:59:10.523987 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc30bb-733a-4a67-bfdf-a0f699aa99e7 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:59:10.525413 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f4bc30bb-733a-4a67-bfdf-a0f699aa99e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:59:10.537150 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc30bb-733a-4a67-bfdf-a0f699aa99e7 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:59:10.540936 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fdb94-12f2-4f70-98ce-3c5b9535c6c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:59:10.541546 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fdb94-12f2-4f70-98ce-3c5b9535c6c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:59:10.541820 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fdb94-12f2-4f70-98ce-3c5b9535c6c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:59:10.542084 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fdb94-12f2-4f70-98ce-3c5b9535c6c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:59:10.544059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f74fdb94-12f2-4f70-98ce-3c5b9535c6c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:59:10.554905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fdb94-12f2-4f70-98ce-3c5b9535c6c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:59:10.556386 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fdb94-12f2-4f70-98ce-3c5b9535c6c0 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:59:10.571798 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1257/2493] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:59:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:59:18.707901 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=32 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:59:20.561085 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=52 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:59:22.469507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debb2f3-aa61-426d-b0cf-e7acb90fae9e service placement] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:59:22.470966 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-6debb2f3-aa61-426d-b0cf-e7acb90fae9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:59:22.546766 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debb2f3-aa61-426d-b0cf-e7acb90fae9e service placement] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:59:22.552894 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98195614-d40e-4761-b0e7-25a350568c10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:59:22.553507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98195614-d40e-4761-b0e7-25a350568c10 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:59:22.553785 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98195614-d40e-4761-b0e7-25a350568c10 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:59:22.554055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98195614-d40e-4761-b0e7-25a350568c10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:59:22.555777 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-98195614-d40e-4761-b0e7-25a350568c10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:59:22.568859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98195614-d40e-4761-b0e7-25a350568c10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:59:22.570426 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98195614-d40e-4761-b0e7-25a350568c10 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:59:22.584269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1237/2494] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:59:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:59:25.261596 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fdb94-12f2-4f70-98ce-3c5b9535c6c0 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 21:59:25.270243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fdb94-12f2-4f70-98ce-3c5b9535c6c0 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 21:59:25.273976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0da31f-128f-467f-84ef-5e248acc01dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 21:59:25.274564 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0da31f-128f-467f-84ef-5e248acc01dd service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 21:59:25.274836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0da31f-128f-467f-84ef-5e248acc01dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 21:59:25.275099 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0da31f-128f-467f-84ef-5e248acc01dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 21:59:25.276789 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-3a0da31f-128f-467f-84ef-5e248acc01dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 21:59:25.337761 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0da31f-128f-467f-84ef-5e248acc01dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 21:59:25.339211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0da31f-128f-467f-84ef-5e248acc01dd service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 21:59:25.349376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1258/2495] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 21:59:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 21:59:32.578690 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=34 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 21:59:35.344312 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=30 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:00:32.342655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98195614-d40e-4761-b0e7-25a350568c10 service nova] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:00:32.358511 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98195614-d40e-4761-b0e7-25a350568c10 service nova] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:00:32.365837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab78c4-ac63-4c6d-a8a5-0f46fef641f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:00:32.366648 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab78c4-ac63-4c6d-a8a5-0f46fef641f1 service nova] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:00:32.367099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab78c4-ac63-4c6d-a8a5-0f46fef641f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:00:32.367532 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab78c4-ac63-4c6d-a8a5-0f46fef641f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:00:32.369660 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-16ab78c4-ac63-4c6d-a8a5-0f46fef641f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:00:32.389786 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab78c4-ac63-4c6d-a8a5-0f46fef641f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:00:32.391710 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab78c4-ac63-4c6d-a8a5-0f46fef641f1 service nova] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:00:32.408588 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1238/2496] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:00:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:00:39.651464 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b52c3-4370-49a1-b9dc-2e0876bef291 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:00:39.651963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b52c3-4370-49a1-b9dc-2e0876bef291 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:00:39.652058 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b52c3-4370-49a1-b9dc-2e0876bef291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:00:39.671215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-127b52c3-4370-49a1-b9dc-2e0876bef291 None None] MFA Rules not processed for user `2346003afe74477496045e9bc72173d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:00:39.672423 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-127b52c3-4370-49a1-b9dc-2e0876bef291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:00:39.725635 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1259/2497] 10.222.0.13 () {58 vars in 967 bytes} [Fri Oct 21 22:00:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 22:00:39.773125 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab78c4-ac63-4c6d-a8a5-0f46fef641f1 service nova] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:00:39.774510 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-16ab78c4-ac63-4c6d-a8a5-0f46fef641f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:00:39.842172 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab78c4-ac63-4c6d-a8a5-0f46fef641f1 service nova] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:00:39.847204 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd885b35-4644-4e9c-a115-0dac55c6ee8c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ce720c8df014f9a8af062138aecd1b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:00:39.847840 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd885b35-4644-4e9c-a115-0dac55c6ee8c service swift] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:00:39.848121 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd885b35-4644-4e9c-a115-0dac55c6ee8c service swift] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:00:39.848396 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd885b35-4644-4e9c-a115-0dac55c6ee8c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:00:39.856940 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd885b35-4644-4e9c-a115-0dac55c6ee8c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:00:39.858512 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd885b35-4644-4e9c-a115-0dac55c6ee8c service swift] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:00:39.866774 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1239/2498] 10.222.0.13 () {60 vars in 1384 bytes} [Fri Oct 21 22:00:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 22:00:43.491444 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b52c3-4370-49a1-b9dc-2e0876bef291 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:00:43.493289 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-127b52c3-4370-49a1-b9dc-2e0876bef291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:00:43.510445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b52c3-4370-49a1-b9dc-2e0876bef291 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:00:43.515690 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195b61b-9282-497f-91d4-14ecc90e586c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:00:43.516299 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195b61b-9282-497f-91d4-14ecc90e586c service placement] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:00:43.516761 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195b61b-9282-497f-91d4-14ecc90e586c service placement] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:00:43.517192 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195b61b-9282-497f-91d4-14ecc90e586c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:00:43.518941 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-5195b61b-9282-497f-91d4-14ecc90e586c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:00:43.533135 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195b61b-9282-497f-91d4-14ecc90e586c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:00:43.534784 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195b61b-9282-497f-91d4-14ecc90e586c service placement] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:00:43.550152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1260/2499] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:00:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:00:46.460111 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd885b35-4644-4e9c-a115-0dac55c6ee8c service swift] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:00:46.461571 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-bd885b35-4644-4e9c-a115-0dac55c6ee8c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:00:46.530247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd885b35-4644-4e9c-a115-0dac55c6ee8c service swift] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:00:46.534570 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae15f2-1eaf-4923-97b4-7844e3c7dfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:00:46.535190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae15f2-1eaf-4923-97b4-7844e3c7dfc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:00:46.535462 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae15f2-1eaf-4923-97b4-7844e3c7dfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:00:46.535758 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae15f2-1eaf-4923-97b4-7844e3c7dfc7 admin admin] PATH_INFO: `/v3/users/933f34b82d5b4355844a57707de417a1` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:00:46.536881 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae15f2-1eaf-4923-97b4-7844e3c7dfc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=933f34b82d5b4355844a57707de417a1)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:00:46.538820 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae15f2-1eaf-4923-97b4-7844e3c7dfc7 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:00:46.587523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5eae15f2-1eaf-4923-97b4-7844e3c7dfc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '933f34b82d5b4355844a57707de417a1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:46.588221 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5eae15f2-1eaf-4923-97b4-7844e3c7dfc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '933f34b82d5b4355844a57707de417a1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:46.600687 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5eae15f2-1eaf-4923-97b4-7844e3c7dfc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '933f34b82d5b4355844a57707de417a1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:46.607901 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5eae15f2-1eaf-4923-97b4-7844e3c7dfc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '933f34b82d5b4355844a57707de417a1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:46.608568 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-5eae15f2-1eaf-4923-97b4-7844e3c7dfc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '933f34b82d5b4355844a57707de417a1'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:46.614850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:00:46.625462 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1240/2500] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 22:00:46 2022] DELETE /identity/v3/users/933f34b82d5b4355844a57707de417a1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:00:46.804433 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195b61b-9282-497f-91d4-14ecc90e586c service placement] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:00:46.815989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195b61b-9282-497f-91d4-14ecc90e586c service placement] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:00:46.872860 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc526160-4c32-4797-a4d8-2924b5feed82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:00:46.873485 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc526160-4c32-4797-a4d8-2924b5feed82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:00:46.873763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc526160-4c32-4797-a4d8-2924b5feed82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:00:46.874033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc526160-4c32-4797-a4d8-2924b5feed82 admin admin] PATH_INFO: `/v3/projects/06e73028dc934238b13f87ed0719038e` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:00:46.875127 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc526160-4c32-4797-a4d8-2924b5feed82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e73028dc934238b13f87ed0719038e)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:00:46.877052 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc526160-4c32-4797-a4d8-2924b5feed82 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:00:46.921737 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-cc526160-4c32-4797-a4d8-2924b5feed82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e73028dc934238b13f87ed0719038e', 'request_id': 'req-cc526160-4c32-4797-a4d8-2924b5feed82'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:46.927676 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-cc526160-4c32-4797-a4d8-2924b5feed82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e73028dc934238b13f87ed0719038e', 'request_id': 'req-cc526160-4c32-4797-a4d8-2924b5feed82'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:46.935326 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-cc526160-4c32-4797-a4d8-2924b5feed82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e73028dc934238b13f87ed0719038e', 'request_id': 'req-cc526160-4c32-4797-a4d8-2924b5feed82'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:46.945843 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-cc526160-4c32-4797-a4d8-2924b5feed82 admin admin] The token cache is being invalidate because project 06e73028dc934238b13f87ed0719038e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 22:00:46.946124 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-cc526160-4c32-4797-a4d8-2924b5feed82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:46.946751 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-cc526160-4c32-4797-a4d8-2924b5feed82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:46.948393 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1261/2501] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 22:00:46 2022] DELETE /identity/v3/projects/06e73028dc934238b13f87ed0719038e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 22:00:46.952573 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae15f2-1eaf-4923-97b4-7844e3c7dfc7 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:00:46.953973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-5eae15f2-1eaf-4923-97b4-7844e3c7dfc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:00:46.966908 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae15f2-1eaf-4923-97b4-7844e3c7dfc7 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:00:47.022390 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9807e74-e344-4004-a2ed-94603bb58b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:00:47.023008 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9807e74-e344-4004-a2ed-94603bb58b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:00:47.023284 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9807e74-e344-4004-a2ed-94603bb58b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:00:47.023550 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9807e74-e344-4004-a2ed-94603bb58b2e admin admin] PATH_INFO: `/v3/users/d8bf07ba081e40e98f17d94749b1cae0` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:00:47.024682 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9807e74-e344-4004-a2ed-94603bb58b2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8bf07ba081e40e98f17d94749b1cae0)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:00:47.026568 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9807e74-e344-4004-a2ed-94603bb58b2e admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:00:47.071216 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a9807e74-e344-4004-a2ed-94603bb58b2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8bf07ba081e40e98f17d94749b1cae0'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:47.071817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a9807e74-e344-4004-a2ed-94603bb58b2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8bf07ba081e40e98f17d94749b1cae0'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:47.083598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a9807e74-e344-4004-a2ed-94603bb58b2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8bf07ba081e40e98f17d94749b1cae0'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:47.090152 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a9807e74-e344-4004-a2ed-94603bb58b2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8bf07ba081e40e98f17d94749b1cae0'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:47.090912 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a9807e74-e344-4004-a2ed-94603bb58b2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8bf07ba081e40e98f17d94749b1cae0'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:47.102864 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1241/2502] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 22:00:46 2022] DELETE /identity/v3/users/d8bf07ba081e40e98f17d94749b1cae0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:00:47.270721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc526160-4c32-4797-a4d8-2924b5feed82 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:00:47.272281 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-cc526160-4c32-4797-a4d8-2924b5feed82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:00:47.286259 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc526160-4c32-4797-a4d8-2924b5feed82 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:00:47.306801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:00:47.347664 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff24d4-577e-4be3-8735-b9773268a030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:00:47.348392 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff24d4-577e-4be3-8735-b9773268a030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:00:47.348677 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff24d4-577e-4be3-8735-b9773268a030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:00:47.348950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff24d4-577e-4be3-8735-b9773268a030 admin admin] PATH_INFO: `/v3/projects/ab91077f61f24a8f85c33e2ea37c6de7` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:00:47.350620 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff24d4-577e-4be3-8735-b9773268a030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab91077f61f24a8f85c33e2ea37c6de7)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:00:47.352647 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff24d4-577e-4be3-8735-b9773268a030 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:00:47.400382 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-87ff24d4-577e-4be3-8735-b9773268a030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab91077f61f24a8f85c33e2ea37c6de7', 'request_id': 'req-87ff24d4-577e-4be3-8735-b9773268a030'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:47.406930 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-87ff24d4-577e-4be3-8735-b9773268a030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab91077f61f24a8f85c33e2ea37c6de7', 'request_id': 'req-87ff24d4-577e-4be3-8735-b9773268a030'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:47.414350 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-87ff24d4-577e-4be3-8735-b9773268a030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab91077f61f24a8f85c33e2ea37c6de7', 'request_id': 'req-87ff24d4-577e-4be3-8735-b9773268a030'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:47.424846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-87ff24d4-577e-4be3-8735-b9773268a030 admin admin] The token cache is being invalidate because project ab91077f61f24a8f85c33e2ea37c6de7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 22:00:47.425130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-87ff24d4-577e-4be3-8735-b9773268a030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:47.425748 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-87ff24d4-577e-4be3-8735-b9773268a030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:00:47.427228 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1262/2503] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 22:00:47 2022] DELETE /identity/v3/projects/ab91077f61f24a8f85c33e2ea37c6de7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:00:57.099522 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:00:57.421803 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:01:24.004870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9807e74-e344-4004-a2ed-94603bb58b2e admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:01:24.006265 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a9807e74-e344-4004-a2ed-94603bb58b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:01:24.120319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9807e74-e344-4004-a2ed-94603bb58b2e admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:01:24.179988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71210a2-7de3-43c1-b3be-437069c6a831 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:01:24.180599 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71210a2-7de3-43c1-b3be-437069c6a831 service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:01:24.180879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71210a2-7de3-43c1-b3be-437069c6a831 service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:01:24.181155 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71210a2-7de3-43c1-b3be-437069c6a831 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:01:24.183594 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a71210a2-7de3-43c1-b3be-437069c6a831 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:01:24.194872 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71210a2-7de3-43c1-b3be-437069c6a831 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:01:24.196465 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71210a2-7de3-43c1-b3be-437069c6a831 service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:01:24.206390 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1242/2504] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:01:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:01:34.202145 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=80 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:01:56.128772 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e9c1d-b6d2-4211-8def-73d2af3111f7 None None] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:01:56.129355 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e9c1d-b6d2-4211-8def-73d2af3111f7 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:01:56.129355 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e9c1d-b6d2-4211-8def-73d2af3111f7 None None] PATH_INFO: `/` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:01:56.130662 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1263/2505] 10.222.0.13 () {54 vars in 921 bytes} [Fri Oct 21 22:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 22:01:56.137139 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa696833-a330-4279-98d5-525e4f7fdb3b None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:01:56.137601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa696833-a330-4279-98d5-525e4f7fdb3b None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:01:56.137715 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa696833-a330-4279-98d5-525e4f7fdb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:01:56.177404 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-fa696833-a330-4279-98d5-525e4f7fdb3b None None] MFA Rules not processed for user `d34dc4e7c5474b6b96977019328d8890`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:01:56.178625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-fa696833-a330-4279-98d5-525e4f7fdb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:01:56.249847 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1243/2506] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Oct 21 22:01:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 22:01:56.262240 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e9c1d-b6d2-4211-8def-73d2af3111f7 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:01:56.264075 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-556e9c1d-b6d2-4211-8def-73d2af3111f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:01:56.278314 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e9c1d-b6d2-4211-8def-73d2af3111f7 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:01:56.282013 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab650375-d7c5-4ab4-92df-89f80133a9c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:01:56.282616 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab650375-d7c5-4ab4-92df-89f80133a9c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:01:56.282894 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab650375-d7c5-4ab4-92df-89f80133a9c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:01:56.283164 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab650375-d7c5-4ab4-92df-89f80133a9c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:01:56.290313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab650375-d7c5-4ab4-92df-89f80133a9c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:01:56.292313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab650375-d7c5-4ab4-92df-89f80133a9c4 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:01:56.300125 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1264/2507] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:01:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:01:56.383406 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fac3eb-a36d-4c5a-ba3b-2872fe9aedd4 None None] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:01:56.383761 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fac3eb-a36d-4c5a-ba3b-2872fe9aedd4 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:01:56.384040 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fac3eb-a36d-4c5a-ba3b-2872fe9aedd4 None None] PATH_INFO: `/` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:01:56.385405 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1244/2508] 10.222.0.13 () {54 vars in 921 bytes} [Fri Oct 21 22:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 22:01:56.390172 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7426b92-d00f-4932-b327-dc48ba0b59af None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:01:56.390452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7426b92-d00f-4932-b327-dc48ba0b59af None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:01:56.390998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7426b92-d00f-4932-b327-dc48ba0b59af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:01:56.427485 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-d7426b92-d00f-4932-b327-dc48ba0b59af None None] MFA Rules not processed for user `12dad9f016a740b98538722e1b723823`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:01:56.428700 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d7426b92-d00f-4932-b327-dc48ba0b59af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:01:56.510471 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1265/2509] 10.222.0.13 () {58 vars in 1018 bytes} [Fri Oct 21 22:01:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:01:56.521231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fac3eb-a36d-4c5a-ba3b-2872fe9aedd4 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:01:56.532688 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fac3eb-a36d-4c5a-ba3b-2872fe9aedd4 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:01:56.536150 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c70de-13b1-49c9-8db1-2275217d02b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:01:56.536742 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c70de-13b1-49c9-8db1-2275217d02b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:01:56.537023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c70de-13b1-49c9-8db1-2275217d02b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:01:56.537297 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c70de-13b1-49c9-8db1-2275217d02b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:01:56.547675 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c70de-13b1-49c9-8db1-2275217d02b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:01:56.549083 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c70de-13b1-49c9-8db1-2275217d02b6 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:01:56.559648 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1245/2510] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:01:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:02:06.507666 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=50 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:06.554975 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=39 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:07.814901 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7426b92-d00f-4932-b327-dc48ba0b59af None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:07.826202 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7426b92-d00f-4932-b327-dc48ba0b59af None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:07.829750 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf3f6a-70f6-4176-bb1a-be8013105c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:07.830355 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf3f6a-70f6-4176-bb1a-be8013105c2f service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:07.830629 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf3f6a-70f6-4176-bb1a-be8013105c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:07.831003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf3f6a-70f6-4176-bb1a-be8013105c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:07.833482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-91bf3f6a-70f6-4176-bb1a-be8013105c2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:07.925706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf3f6a-70f6-4176-bb1a-be8013105c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:07.927486 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf3f6a-70f6-4176-bb1a-be8013105c2f service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:07.938970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1266/2511] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:02:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:02:11.053506 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c70de-13b1-49c9-8db1-2275217d02b6 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:11.055154 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-df4c70de-13b1-49c9-8db1-2275217d02b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:11.070065 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c70de-13b1-49c9-8db1-2275217d02b6 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:11.075514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd4303-7bf4-4f31-b12b-43b9c26112fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:11.076185 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd4303-7bf4-4f31-b12b-43b9c26112fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:11.076468 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd4303-7bf4-4f31-b12b-43b9c26112fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:11.076740 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd4303-7bf4-4f31-b12b-43b9c26112fe admin admin] PATH_INFO: `/v3/users/00374760bfa648e48dedae8aadb61c8f` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:11.077931 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd4303-7bf4-4f31-b12b-43b9c26112fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=00374760bfa648e48dedae8aadb61c8f)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:11.079908 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd4303-7bf4-4f31-b12b-43b9c26112fe admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:11.133750 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9acd4303-7bf4-4f31-b12b-43b9c26112fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00374760bfa648e48dedae8aadb61c8f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:11.134399 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9acd4303-7bf4-4f31-b12b-43b9c26112fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00374760bfa648e48dedae8aadb61c8f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:11.146276 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9acd4303-7bf4-4f31-b12b-43b9c26112fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00374760bfa648e48dedae8aadb61c8f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:11.154581 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9acd4303-7bf4-4f31-b12b-43b9c26112fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00374760bfa648e48dedae8aadb61c8f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:11.155234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9acd4303-7bf4-4f31-b12b-43b9c26112fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00374760bfa648e48dedae8aadb61c8f'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:11.167946 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1246/2512] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 22:02:11 2022] DELETE /identity/v3/users/00374760bfa648e48dedae8aadb61c8f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:11.337286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf3f6a-70f6-4176-bb1a-be8013105c2f service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:11.348155 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf3f6a-70f6-4176-bb1a-be8013105c2f service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:11.409529 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1547b-2c31-4c4d-a97e-36ca94b5f01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:11.410156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1547b-2c31-4c4d-a97e-36ca94b5f01c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:11.410435 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1547b-2c31-4c4d-a97e-36ca94b5f01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:11.410708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1547b-2c31-4c4d-a97e-36ca94b5f01c admin admin] PATH_INFO: `/v3/projects/cdd4b4b0f51940ee829562303eca9ce9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:11.411851 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1547b-2c31-4c4d-a97e-36ca94b5f01c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdd4b4b0f51940ee829562303eca9ce9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:11.413749 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1547b-2c31-4c4d-a97e-36ca94b5f01c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:11.464129 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e2c1547b-2c31-4c4d-a97e-36ca94b5f01c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdd4b4b0f51940ee829562303eca9ce9', 'request_id': 'req-e2c1547b-2c31-4c4d-a97e-36ca94b5f01c'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:11.470880 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e2c1547b-2c31-4c4d-a97e-36ca94b5f01c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdd4b4b0f51940ee829562303eca9ce9', 'request_id': 'req-e2c1547b-2c31-4c4d-a97e-36ca94b5f01c'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:11.478574 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e2c1547b-2c31-4c4d-a97e-36ca94b5f01c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdd4b4b0f51940ee829562303eca9ce9', 'request_id': 'req-e2c1547b-2c31-4c4d-a97e-36ca94b5f01c'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:11.489122 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e2c1547b-2c31-4c4d-a97e-36ca94b5f01c admin admin] The token cache is being invalidate because project cdd4b4b0f51940ee829562303eca9ce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 22:02:11.489483 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e2c1547b-2c31-4c4d-a97e-36ca94b5f01c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:11.490029 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-e2c1547b-2c31-4c4d-a97e-36ca94b5f01c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:11.492856 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1267/2513] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 22:02:11 2022] DELETE /identity/v3/projects/cdd4b4b0f51940ee829562303eca9ce9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:11.498654 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b017dc-b8b3-4820-8b37-19cf0bce89b5 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:11.498986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b017dc-b8b3-4820-8b37-19cf0bce89b5 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:11.499262 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b017dc-b8b3-4820-8b37-19cf0bce89b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:11.519190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-d7b017dc-b8b3-4820-8b37-19cf0bce89b5 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:11.520439 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d7b017dc-b8b3-4820-8b37-19cf0bce89b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:11.580082 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1247/2514] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:11.587517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cc0b3-0a99-4117-8ba5-ab1f20d55a5a None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:11.588423 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cc0b3-0a99-4117-8ba5-ab1f20d55a5a None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:11.588707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cc0b3-0a99-4117-8ba5-ab1f20d55a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:11.608404 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-ed2cc0b3-0a99-4117-8ba5-ab1f20d55a5a None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:11.609593 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-ed2cc0b3-0a99-4117-8ba5-ab1f20d55a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:11.617411 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1268/2515] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:11.621668 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b017dc-b8b3-4820-8b37-19cf0bce89b5 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:11.631447 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b017dc-b8b3-4820-8b37-19cf0bce89b5 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:11.636569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e30986-ffe7-4494-ab4d-c9e5c61c9916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:11.637158 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e30986-ffe7-4494-ab4d-c9e5c61c9916 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:11.637437 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e30986-ffe7-4494-ab4d-c9e5c61c9916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:11.637705 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e30986-ffe7-4494-ab4d-c9e5c61c9916 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:11.638336 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e30986-ffe7-4494-ab4d-c9e5c61c9916 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 22:02:11.638639 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e30986-ffe7-4494-ab4d-c9e5c61c9916 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:11.640238 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e30986-ffe7-4494-ab4d-c9e5c61c9916 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:11.652354 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1248/2516] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 22:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 22:02:11.656420 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cc0b3-0a99-4117-8ba5-ab1f20d55a5a None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:11.665378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cc0b3-0a99-4117-8ba5-ab1f20d55a5a None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:11.670018 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d03d2-fd30-4d52-bbbd-ce70964f795d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:11.670603 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d03d2-fd30-4d52-bbbd-ce70964f795d admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:11.670877 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d03d2-fd30-4d52-bbbd-ce70964f795d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:11.671146 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d03d2-fd30-4d52-bbbd-ce70964f795d admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:11.671783 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d03d2-fd30-4d52-bbbd-ce70964f795d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:11.673669 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d03d2-fd30-4d52-bbbd-ce70964f795d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:11.688712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:11.702211 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1269/2517] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 22:02:11 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:11.706167 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e30986-ffe7-4494-ab4d-c9e5c61c9916 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:11.716656 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e30986-ffe7-4494-ab4d-c9e5c61c9916 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:11.772430 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f847858-1b14-469e-908f-f9429de98049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:11.772995 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f847858-1b14-469e-908f-f9429de98049 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:11.773269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f847858-1b14-469e-908f-f9429de98049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:11.773533 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f847858-1b14-469e-908f-f9429de98049 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:11.774157 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f847858-1b14-469e-908f-f9429de98049 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:11.776126 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f847858-1b14-469e-908f-f9429de98049 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:11.784628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:11.800150 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1249/2518] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 22:02:11 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:11.804439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d03d2-fd30-4d52-bbbd-ce70964f795d admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:11.814971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d03d2-fd30-4d52-bbbd-ce70964f795d admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:11.819414 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfedc3-5709-4c69-b6a7-ec02e9c0837d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:11.820044 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfedc3-5709-4c69-b6a7-ec02e9c0837d admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:11.820318 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfedc3-5709-4c69-b6a7-ec02e9c0837d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:11.820585 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfedc3-5709-4c69-b6a7-ec02e9c0837d admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:11.821210 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccfedc3-5709-4c69-b6a7-ec02e9c0837d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:11.822792 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccfedc3-5709-4c69-b6a7-ec02e9c0837d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:11.833594 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1270/2519] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:11.837347 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f847858-1b14-469e-908f-f9429de98049 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:11.847655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f847858-1b14-469e-908f-f9429de98049 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:11.852157 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d371f-0f08-4004-9e07-f9786f79b272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:11.852740 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d371f-0f08-4004-9e07-f9786f79b272 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:11.853020 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d371f-0f08-4004-9e07-f9786f79b272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:11.853292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d371f-0f08-4004-9e07-f9786f79b272 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:11.853917 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d371f-0f08-4004-9e07-f9786f79b272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:11.855589 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d371f-0f08-4004-9e07-f9786f79b272 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:11.865829 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1250/2520] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 22:02:11.869796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfedc3-5709-4c69-b6a7-ec02e9c0837d admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:11.880150 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfedc3-5709-4c69-b6a7-ec02e9c0837d admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:11.884980 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f2e03-8eb4-44a7-9aba-33ff97d3f7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:11.885621 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f2e03-8eb4-44a7-9aba-33ff97d3f7d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:11.886500 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f2e03-8eb4-44a7-9aba-33ff97d3f7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:11.886770 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f2e03-8eb4-44a7-9aba-33ff97d3f7d1 admin admin] PATH_INFO: `/v3/projects/448875d26053424bb961494ec5683d61/users/8912b0b409fd421293aa693b43d23836/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:11.900905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f2e03-8eb4-44a7-9aba-33ff97d3f7d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=448875d26053424bb961494ec5683d61, user_id=8912b0b409fd421293aa693b43d23836, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:11.903550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f2e03-8eb4-44a7-9aba-33ff97d3f7d1 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:11.916041 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1271/2521] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 22:02:11 2022] PUT /identity/v3/projects/448875d26053424bb961494ec5683d61/users/8912b0b409fd421293aa693b43d23836/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:11.927753 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d1c69-f98c-4f2f-b35e-c81d550b5e7a None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:11.928039 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d1c69-f98c-4f2f-b35e-c81d550b5e7a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:11.928337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d1c69-f98c-4f2f-b35e-c81d550b5e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:11.956638 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-495d1c69-f98c-4f2f-b35e-c81d550b5e7a None None] MFA Rules not processed for user `8912b0b409fd421293aa693b43d23836`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:11.957853 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-495d1c69-f98c-4f2f-b35e-c81d550b5e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:12.019990 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1251/2522] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 22:02:12.024474 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f2e03-8eb4-44a7-9aba-33ff97d3f7d1 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:12.086383 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f2e03-8eb4-44a7-9aba-33ff97d3f7d1 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:12.090999 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e4f54-15c2-48f9-962c-1df979b4ed92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:12.091621 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e4f54-15c2-48f9-962c-1df979b4ed92 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:12.091898 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e4f54-15c2-48f9-962c-1df979b4ed92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:12.092164 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e4f54-15c2-48f9-962c-1df979b4ed92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:12.092775 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e4f54-15c2-48f9-962c-1df979b4ed92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:12.094659 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e4f54-15c2-48f9-962c-1df979b4ed92 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:12.119638 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1272/2523] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 22:02:12 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:12.124260 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d1c69-f98c-4f2f-b35e-c81d550b5e7a None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:12.135281 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d1c69-f98c-4f2f-b35e-c81d550b5e7a None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:12.189928 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d61fa-0d8d-4e69-8299-9b1490957111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:12.190528 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d61fa-0d8d-4e69-8299-9b1490957111 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:12.190802 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d61fa-0d8d-4e69-8299-9b1490957111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:12.191071 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d61fa-0d8d-4e69-8299-9b1490957111 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:12.191713 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d61fa-0d8d-4e69-8299-9b1490957111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:12.193592 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d61fa-0d8d-4e69-8299-9b1490957111 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:12.213692 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1252/2524] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 22:02:12 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:12.217659 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e4f54-15c2-48f9-962c-1df979b4ed92 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:12.227835 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e4f54-15c2-48f9-962c-1df979b4ed92 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:12.232860 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b03936-9464-48a2-bca7-243a151a7e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:12.233445 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b03936-9464-48a2-bca7-243a151a7e61 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:12.233721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b03936-9464-48a2-bca7-243a151a7e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:12.233989 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b03936-9464-48a2-bca7-243a151a7e61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:12.234619 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b03936-9464-48a2-bca7-243a151a7e61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:12.236234 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b03936-9464-48a2-bca7-243a151a7e61 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:12.247407 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1273/2525] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:12.251358 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d61fa-0d8d-4e69-8299-9b1490957111 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:12.261288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d61fa-0d8d-4e69-8299-9b1490957111 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:12.265880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11daaca-5f92-4e87-822a-483cfb23d8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:12.266520 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11daaca-5f92-4e87-822a-483cfb23d8fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:12.266798 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11daaca-5f92-4e87-822a-483cfb23d8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:12.267069 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11daaca-5f92-4e87-822a-483cfb23d8fe admin admin] PATH_INFO: `/v3/projects/e204cfdef2f54a77bd10ed01755dde3f/users/16e2c97dc3b7434b919ad03b6ee9c573/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:12.281615 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11daaca-5f92-4e87-822a-483cfb23d8fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e204cfdef2f54a77bd10ed01755dde3f, user_id=16e2c97dc3b7434b919ad03b6ee9c573, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:12.284343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11daaca-5f92-4e87-822a-483cfb23d8fe admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:12.297755 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1253/2526] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 22:02:12 2022] PUT /identity/v3/projects/e204cfdef2f54a77bd10ed01755dde3f/users/16e2c97dc3b7434b919ad03b6ee9c573/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:12.301581 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b03936-9464-48a2-bca7-243a151a7e61 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:12.311907 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b03936-9464-48a2-bca7-243a151a7e61 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:12.365618 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a627759-4067-4a09-96e6-f346824ead4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:12.366251 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a627759-4067-4a09-96e6-f346824ead4c admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:12.366478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a627759-4067-4a09-96e6-f346824ead4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:12.366747 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a627759-4067-4a09-96e6-f346824ead4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:12.367376 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a627759-4067-4a09-96e6-f346824ead4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:12.369025 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a627759-4067-4a09-96e6-f346824ead4c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:12.379113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1274/2527] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:12.383516 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11daaca-5f92-4e87-822a-483cfb23d8fe admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:12.393876 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11daaca-5f92-4e87-822a-483cfb23d8fe admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:12.398465 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe7b19-a446-47aa-ab56-67d6ecce01b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:12.399107 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe7b19-a446-47aa-ab56-67d6ecce01b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:12.399387 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe7b19-a446-47aa-ab56-67d6ecce01b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:12.399697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe7b19-a446-47aa-ab56-67d6ecce01b2 admin admin] PATH_INFO: `/v3/projects/e204cfdef2f54a77bd10ed01755dde3f/users/16e2c97dc3b7434b919ad03b6ee9c573/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:12.401612 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe7b19-a446-47aa-ab56-67d6ecce01b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e204cfdef2f54a77bd10ed01755dde3f, user_id=16e2c97dc3b7434b919ad03b6ee9c573, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:12.404283 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe7b19-a446-47aa-ab56-67d6ecce01b2 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:12.414508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1254/2528] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 22:02:12 2022] PUT /identity/v3/projects/e204cfdef2f54a77bd10ed01755dde3f/users/16e2c97dc3b7434b919ad03b6ee9c573/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:12.424603 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda417e9-9962-4c66-befa-7251b7a9ab8d None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:12.424890 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda417e9-9962-4c66-befa-7251b7a9ab8d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:12.425163 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda417e9-9962-4c66-befa-7251b7a9ab8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:12.452532 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-eda417e9-9962-4c66-befa-7251b7a9ab8d None None] MFA Rules not processed for user `16e2c97dc3b7434b919ad03b6ee9c573`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:12.453732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-eda417e9-9962-4c66-befa-7251b7a9ab8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:12.471760 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:12.521497 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1275/2529] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:12 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:12.533326 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe7b19-a446-47aa-ab56-67d6ecce01b2 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:12.534974 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8abe7b19-a446-47aa-ab56-67d6ecce01b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:12.559554 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:12.606920 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe7b19-a446-47aa-ab56-67d6ecce01b2 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:12.611030 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d24d7-c728-40e7-9429-609ea06da561 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:12.611647 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d24d7-c728-40e7-9429-609ea06da561 service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:12.611923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d24d7-c728-40e7-9429-609ea06da561 service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:12.612191 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d24d7-c728-40e7-9429-609ea06da561 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:12.621119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d24d7-c728-40e7-9429-609ea06da561 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:12.622516 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d24d7-c728-40e7-9429-609ea06da561 service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:12.631989 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1255/2530] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:02:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:02:12.706180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda417e9-9962-4c66-befa-7251b7a9ab8d None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:12.716525 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda417e9-9962-4c66-befa-7251b7a9ab8d None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:12.720067 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fddb4a-11d7-45fb-9a2e-59f0f174ba0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:12.720671 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fddb4a-11d7-45fb-9a2e-59f0f174ba0b service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:12.720954 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fddb4a-11d7-45fb-9a2e-59f0f174ba0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:12.721230 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fddb4a-11d7-45fb-9a2e-59f0f174ba0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:12.778172 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fddb4a-11d7-45fb-9a2e-59f0f174ba0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:12.779627 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fddb4a-11d7-45fb-9a2e-59f0f174ba0b service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:12.788415 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1276/2531] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:02:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:02:22.627561 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=22 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:22.784386 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=48 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:25.131466 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d24d7-c728-40e7-9429-609ea06da561 service cinder] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:25.198208 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d24d7-c728-40e7-9429-609ea06da561 service cinder] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:25.203299 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25282ba6-ea5d-43b1-918f-584fabbd5f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:25.204066 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25282ba6-ea5d-43b1-918f-584fabbd5f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:25.204406 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25282ba6-ea5d-43b1-918f-584fabbd5f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:25.204680 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25282ba6-ea5d-43b1-918f-584fabbd5f30 admin admin] PATH_INFO: `/v3/users/8912b0b409fd421293aa693b43d23836` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:25.205915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25282ba6-ea5d-43b1-918f-584fabbd5f30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8912b0b409fd421293aa693b43d23836)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:25.207850 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25282ba6-ea5d-43b1-918f-584fabbd5f30 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:25.260726 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-25282ba6-ea5d-43b1-918f-584fabbd5f30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8912b0b409fd421293aa693b43d23836'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:25.261309 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-25282ba6-ea5d-43b1-918f-584fabbd5f30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8912b0b409fd421293aa693b43d23836'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:25.274839 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-25282ba6-ea5d-43b1-918f-584fabbd5f30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8912b0b409fd421293aa693b43d23836'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:25.285192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-25282ba6-ea5d-43b1-918f-584fabbd5f30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8912b0b409fd421293aa693b43d23836'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:25.286468 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-25282ba6-ea5d-43b1-918f-584fabbd5f30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8912b0b409fd421293aa693b43d23836'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:25.301604 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1256/2532] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 22:02:25 2022] DELETE /identity/v3/users/8912b0b409fd421293aa693b43d23836 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 22:02:25.306011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7ef27-1366-489c-90cd-0b046fc052e2 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:25.306471 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7ef27-1366-489c-90cd-0b046fc052e2 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:25.306573 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7ef27-1366-489c-90cd-0b046fc052e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:25.336733 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-adc7ef27-1366-489c-90cd-0b046fc052e2 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:25.338020 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-adc7ef27-1366-489c-90cd-0b046fc052e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:25.396905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1277/2533] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:25.407125 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25282ba6-ea5d-43b1-918f-584fabbd5f30 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:25.408569 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-25282ba6-ea5d-43b1-918f-584fabbd5f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:25.464915 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25282ba6-ea5d-43b1-918f-584fabbd5f30 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:25.468551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf702c55-c953-4103-a35a-b2c402df6af7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:25.469152 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf702c55-c953-4103-a35a-b2c402df6af7 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:25.469429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf702c55-c953-4103-a35a-b2c402df6af7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:25.469701 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf702c55-c953-4103-a35a-b2c402df6af7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:25.478606 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf702c55-c953-4103-a35a-b2c402df6af7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:25.480686 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf702c55-c953-4103-a35a-b2c402df6af7 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:25.489920 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1257/2534] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:02:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:02:25.753825 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc7ef27-1366-489c-90cd-0b046fc052e2 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:25.755247 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-adc7ef27-1366-489c-90cd-0b046fc052e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:25.768345 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc7ef27-1366-489c-90cd-0b046fc052e2 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:25.773243 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0d582-b625-4f6a-b054-91b39a9f4f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:25.773906 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0d582-b625-4f6a-b054-91b39a9f4f66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:25.774181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0d582-b625-4f6a-b054-91b39a9f4f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:25.774451 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0d582-b625-4f6a-b054-91b39a9f4f66 admin admin] PATH_INFO: `/v3/projects/448875d26053424bb961494ec5683d61` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:25.775596 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0d582-b625-4f6a-b054-91b39a9f4f66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=448875d26053424bb961494ec5683d61)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:25.778286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0d582-b625-4f6a-b054-91b39a9f4f66 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:25.825709 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0cd0d582-b625-4f6a-b054-91b39a9f4f66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '448875d26053424bb961494ec5683d61', 'request_id': 'req-0cd0d582-b625-4f6a-b054-91b39a9f4f66'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:25.831706 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0cd0d582-b625-4f6a-b054-91b39a9f4f66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '448875d26053424bb961494ec5683d61', 'request_id': 'req-0cd0d582-b625-4f6a-b054-91b39a9f4f66'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:25.838971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0cd0d582-b625-4f6a-b054-91b39a9f4f66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '448875d26053424bb961494ec5683d61', 'request_id': 'req-0cd0d582-b625-4f6a-b054-91b39a9f4f66'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:25.849181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0cd0d582-b625-4f6a-b054-91b39a9f4f66 admin admin] The token cache is being invalidate because project 448875d26053424bb961494ec5683d61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 22:02:25.849464 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0cd0d582-b625-4f6a-b054-91b39a9f4f66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:25.850087 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-0cd0d582-b625-4f6a-b054-91b39a9f4f66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:25.851605 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1278/2535] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 22:02:25 2022] DELETE /identity/v3/projects/448875d26053424bb961494ec5683d61 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:25.856323 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf702c55-c953-4103-a35a-b2c402df6af7 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:25.865791 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf702c55-c953-4103-a35a-b2c402df6af7 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:25.884139 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:25.923581 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d26dd1-7652-4925-8cde-bf96ff2af4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:25.924269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d26dd1-7652-4925-8cde-bf96ff2af4e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:25.924548 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d26dd1-7652-4925-8cde-bf96ff2af4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:25.924820 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d26dd1-7652-4925-8cde-bf96ff2af4e2 admin admin] PATH_INFO: `/v3/users/16e2c97dc3b7434b919ad03b6ee9c573` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:25.925943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d26dd1-7652-4925-8cde-bf96ff2af4e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16e2c97dc3b7434b919ad03b6ee9c573)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:25.927913 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d26dd1-7652-4925-8cde-bf96ff2af4e2 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:25.971380 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-44d26dd1-7652-4925-8cde-bf96ff2af4e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16e2c97dc3b7434b919ad03b6ee9c573'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:25.972056 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-44d26dd1-7652-4925-8cde-bf96ff2af4e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16e2c97dc3b7434b919ad03b6ee9c573'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:25.984150 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-44d26dd1-7652-4925-8cde-bf96ff2af4e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16e2c97dc3b7434b919ad03b6ee9c573'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:25.991248 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-44d26dd1-7652-4925-8cde-bf96ff2af4e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16e2c97dc3b7434b919ad03b6ee9c573'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:25.991879 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-44d26dd1-7652-4925-8cde-bf96ff2af4e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16e2c97dc3b7434b919ad03b6ee9c573'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:26.004739 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1258/2536] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 22:02:25 2022] DELETE /identity/v3/users/16e2c97dc3b7434b919ad03b6ee9c573 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:26.270458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0d582-b625-4f6a-b054-91b39a9f4f66 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:26.271910 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-0cd0d582-b625-4f6a-b054-91b39a9f4f66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:26.286732 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0d582-b625-4f6a-b054-91b39a9f4f66 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:26.343646 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8acde5-3290-41a5-ad50-2cc7e5f7f78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:26.344334 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8acde5-3290-41a5-ad50-2cc7e5f7f78e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:26.344544 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8acde5-3290-41a5-ad50-2cc7e5f7f78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:26.344812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8acde5-3290-41a5-ad50-2cc7e5f7f78e admin admin] PATH_INFO: `/v3/projects/e204cfdef2f54a77bd10ed01755dde3f` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:26.345915 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8acde5-3290-41a5-ad50-2cc7e5f7f78e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e204cfdef2f54a77bd10ed01755dde3f)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:26.347916 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8acde5-3290-41a5-ad50-2cc7e5f7f78e admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:26.364799 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:26.395292 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-5e8acde5-3290-41a5-ad50-2cc7e5f7f78e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e204cfdef2f54a77bd10ed01755dde3f', 'request_id': 'req-5e8acde5-3290-41a5-ad50-2cc7e5f7f78e'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:26.402038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-5e8acde5-3290-41a5-ad50-2cc7e5f7f78e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e204cfdef2f54a77bd10ed01755dde3f', 'request_id': 'req-5e8acde5-3290-41a5-ad50-2cc7e5f7f78e'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:26.408569 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-5e8acde5-3290-41a5-ad50-2cc7e5f7f78e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e204cfdef2f54a77bd10ed01755dde3f', 'request_id': 'req-5e8acde5-3290-41a5-ad50-2cc7e5f7f78e'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:26.419816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-5e8acde5-3290-41a5-ad50-2cc7e5f7f78e admin admin] The token cache is being invalidate because project e204cfdef2f54a77bd10ed01755dde3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 22:02:26.421432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-5e8acde5-3290-41a5-ad50-2cc7e5f7f78e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:26.422084 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-5e8acde5-3290-41a5-ad50-2cc7e5f7f78e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:26.424661 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1279/2537] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 22:02:26 2022] DELETE /identity/v3/projects/e204cfdef2f54a77bd10ed01755dde3f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:26.429570 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f44df-df0c-48f6-8564-0ce699ddee54 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:26.430055 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f44df-df0c-48f6-8564-0ce699ddee54 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:26.430144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f44df-df0c-48f6-8564-0ce699ddee54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:26.450163 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-3c3f44df-df0c-48f6-8564-0ce699ddee54 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:26.451361 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-3c3f44df-df0c-48f6-8564-0ce699ddee54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:26.508311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1259/2538] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:26.516458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a4bf0-d6b2-4d0c-8ddc-c652f4665a99 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:26.516741 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a4bf0-d6b2-4d0c-8ddc-c652f4665a99 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:26.517012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a4bf0-d6b2-4d0c-8ddc-c652f4665a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:26.536737 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-565a4bf0-d6b2-4d0c-8ddc-c652f4665a99 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:26.537924 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-565a4bf0-d6b2-4d0c-8ddc-c652f4665a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:26.545548 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1280/2539] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:26.549523 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f44df-df0c-48f6-8564-0ce699ddee54 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:26.559075 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f44df-df0c-48f6-8564-0ce699ddee54 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:26.563698 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e78854-d1ed-4e5f-98f4-219745912313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:26.564296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e78854-d1ed-4e5f-98f4-219745912313 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:26.564575 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e78854-d1ed-4e5f-98f4-219745912313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:26.564844 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e78854-d1ed-4e5f-98f4-219745912313 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:26.565493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e78854-d1ed-4e5f-98f4-219745912313 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 22:02:26.565784 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e78854-d1ed-4e5f-98f4-219745912313 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:26.567357 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e78854-d1ed-4e5f-98f4-219745912313 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:26.579836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1260/2540] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 22:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 22:02:26.583912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a4bf0-d6b2-4d0c-8ddc-c652f4665a99 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:26.593526 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a4bf0-d6b2-4d0c-8ddc-c652f4665a99 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:26.598055 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad28a63-a764-4ca9-8955-efc73b39039e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:26.598645 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad28a63-a764-4ca9-8955-efc73b39039e admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:26.598918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad28a63-a764-4ca9-8955-efc73b39039e admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:26.599181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad28a63-a764-4ca9-8955-efc73b39039e admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:26.599825 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad28a63-a764-4ca9-8955-efc73b39039e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:26.601743 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad28a63-a764-4ca9-8955-efc73b39039e admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:26.629206 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1281/2541] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 22:02:26 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:26.633461 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e78854-d1ed-4e5f-98f4-219745912313 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:26.643952 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e78854-d1ed-4e5f-98f4-219745912313 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:26.669801 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:26.701259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f85c0-7e2d-4672-a94b-eb16a4111028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:26.701862 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f85c0-7e2d-4672-a94b-eb16a4111028 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:26.702144 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f85c0-7e2d-4672-a94b-eb16a4111028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:26.702421 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f85c0-7e2d-4672-a94b-eb16a4111028 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:26.703041 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f85c0-7e2d-4672-a94b-eb16a4111028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:26.705001 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f85c0-7e2d-4672-a94b-eb16a4111028 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:26.729680 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1261/2542] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 22:02:26 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:26.730830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad28a63-a764-4ca9-8955-efc73b39039e admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:26.741470 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad28a63-a764-4ca9-8955-efc73b39039e admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:26.746035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3b29f-a482-493d-93c8-d05777b6aa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:26.746636 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3b29f-a482-493d-93c8-d05777b6aa3f admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:26.746917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3b29f-a482-493d-93c8-d05777b6aa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:26.747190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3b29f-a482-493d-93c8-d05777b6aa3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:26.747850 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc3b29f-a482-493d-93c8-d05777b6aa3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:26.749453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc3b29f-a482-493d-93c8-d05777b6aa3f admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:26.759403 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1282/2543] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 22:02:26.763778 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f85c0-7e2d-4672-a94b-eb16a4111028 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:26.773456 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f85c0-7e2d-4672-a94b-eb16a4111028 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:26.778597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0997e-7cb4-404d-bc10-bec6729aff51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:26.779184 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0997e-7cb4-404d-bc10-bec6729aff51 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:26.779463 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0997e-7cb4-404d-bc10-bec6729aff51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:26.779764 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0997e-7cb4-404d-bc10-bec6729aff51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:26.780393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0997e-7cb4-404d-bc10-bec6729aff51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:26.781962 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0997e-7cb4-404d-bc10-bec6729aff51 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:26.792583 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1262/2544] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:26.796375 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3b29f-a482-493d-93c8-d05777b6aa3f admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:26.806013 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3b29f-a482-493d-93c8-d05777b6aa3f admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:26.810998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1a836-1ba1-4d75-ac6e-76792f23c427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:26.811660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1a836-1ba1-4d75-ac6e-76792f23c427 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:26.811936 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1a836-1ba1-4d75-ac6e-76792f23c427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:26.812202 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1a836-1ba1-4d75-ac6e-76792f23c427 admin admin] PATH_INFO: `/v3/projects/b69e545b13eb4c599f21591f628b7ecd/users/8b49ef30c2a348018bdbcba5ba13ef56/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:26.826111 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de1a836-1ba1-4d75-ac6e-76792f23c427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b69e545b13eb4c599f21591f628b7ecd, user_id=8b49ef30c2a348018bdbcba5ba13ef56, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:26.828744 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de1a836-1ba1-4d75-ac6e-76792f23c427 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:26.841917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1283/2545] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 22:02:26 2022] PUT /identity/v3/projects/b69e545b13eb4c599f21591f628b7ecd/users/8b49ef30c2a348018bdbcba5ba13ef56/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 22:02:26.852815 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b86935-29ea-4a41-b071-7a34a3c6c6eb None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:26.853099 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b86935-29ea-4a41-b071-7a34a3c6c6eb None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:26.853373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b86935-29ea-4a41-b071-7a34a3c6c6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:26.881920 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-02b86935-29ea-4a41-b071-7a34a3c6c6eb None None] MFA Rules not processed for user `8b49ef30c2a348018bdbcba5ba13ef56`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:26.883115 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-02b86935-29ea-4a41-b071-7a34a3c6c6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:26.945502 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1263/2546] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:26.949873 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1a836-1ba1-4d75-ac6e-76792f23c427 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:27.009689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1a836-1ba1-4d75-ac6e-76792f23c427 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:27.014321 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b2c17-78fa-4315-8d75-b04371e38425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:27.014917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b2c17-78fa-4315-8d75-b04371e38425 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:27.015198 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b2c17-78fa-4315-8d75-b04371e38425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:27.015472 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b2c17-78fa-4315-8d75-b04371e38425 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:27.016134 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b2c17-78fa-4315-8d75-b04371e38425 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:27.018072 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b2c17-78fa-4315-8d75-b04371e38425 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:27.044633 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1284/2547] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 22:02:26 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 22:02:27.048572 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b86935-29ea-4a41-b071-7a34a3c6c6eb None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:27.059053 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b86935-29ea-4a41-b071-7a34a3c6c6eb None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:27.114158 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef199cf7-bcc3-4f7b-9690-6ce58ae7b7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:27.114770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef199cf7-bcc3-4f7b-9690-6ce58ae7b7cb admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:27.115051 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef199cf7-bcc3-4f7b-9690-6ce58ae7b7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:27.115304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef199cf7-bcc3-4f7b-9690-6ce58ae7b7cb admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:27.115949 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef199cf7-bcc3-4f7b-9690-6ce58ae7b7cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:27.117872 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef199cf7-bcc3-4f7b-9690-6ce58ae7b7cb admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:27.138683 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1264/2548] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 22:02:27 2022] POST /identity/v3/users => generated 299 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 22:02:27.142424 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b2c17-78fa-4315-8d75-b04371e38425 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:27.153388 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b2c17-78fa-4315-8d75-b04371e38425 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:27.158079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763c2ad-adb9-4f4b-b497-222aa4d098c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:27.158671 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763c2ad-adb9-4f4b-b497-222aa4d098c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:27.158948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763c2ad-adb9-4f4b-b497-222aa4d098c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:27.159213 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763c2ad-adb9-4f4b-b497-222aa4d098c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:27.159870 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f763c2ad-adb9-4f4b-b497-222aa4d098c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:27.161467 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f763c2ad-adb9-4f4b-b497-222aa4d098c8 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:27.173229 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1285/2549] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:27.177131 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef199cf7-bcc3-4f7b-9690-6ce58ae7b7cb admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:27.186817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef199cf7-bcc3-4f7b-9690-6ce58ae7b7cb admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:27.191557 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea78c4-594c-4c8c-9f3a-fddf2e96ac44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:27.192234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea78c4-594c-4c8c-9f3a-fddf2e96ac44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:27.192517 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea78c4-594c-4c8c-9f3a-fddf2e96ac44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:27.192795 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea78c4-594c-4c8c-9f3a-fddf2e96ac44 admin admin] PATH_INFO: `/v3/projects/215b29e74d4e41118d1a0d11742d849b/users/6101d4540a3e44a197b0c007ec8c61e5/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:27.207696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea78c4-594c-4c8c-9f3a-fddf2e96ac44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=215b29e74d4e41118d1a0d11742d849b, user_id=6101d4540a3e44a197b0c007ec8c61e5, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:27.210320 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea78c4-594c-4c8c-9f3a-fddf2e96ac44 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:27.223160 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1265/2550] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 22:02:27 2022] PUT /identity/v3/projects/215b29e74d4e41118d1a0d11742d849b/users/6101d4540a3e44a197b0c007ec8c61e5/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:27.227007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763c2ad-adb9-4f4b-b497-222aa4d098c8 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:27.236884 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763c2ad-adb9-4f4b-b497-222aa4d098c8 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:27.248708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:27.294213 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f3e5b-7049-484c-8bbd-1b2086f44314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:27.294808 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f3e5b-7049-484c-8bbd-1b2086f44314 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:27.295083 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f3e5b-7049-484c-8bbd-1b2086f44314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:27.295358 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f3e5b-7049-484c-8bbd-1b2086f44314 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:27.296074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f3e5b-7049-484c-8bbd-1b2086f44314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:27.297741 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f3e5b-7049-484c-8bbd-1b2086f44314 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:27.307095 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1286/2551] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:27.311177 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea78c4-594c-4c8c-9f3a-fddf2e96ac44 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:27.316496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:27.323270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea78c4-594c-4c8c-9f3a-fddf2e96ac44 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:27.328048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d1846-18f2-4689-9719-aa0635c42463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:27.328691 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d1846-18f2-4689-9719-aa0635c42463 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:27.328973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d1846-18f2-4689-9719-aa0635c42463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:27.329243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d1846-18f2-4689-9719-aa0635c42463 admin admin] PATH_INFO: `/v3/projects/215b29e74d4e41118d1a0d11742d849b/users/6101d4540a3e44a197b0c007ec8c61e5/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:27.331141 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d1846-18f2-4689-9719-aa0635c42463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=215b29e74d4e41118d1a0d11742d849b, user_id=6101d4540a3e44a197b0c007ec8c61e5, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:27.333824 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d1846-18f2-4689-9719-aa0635c42463 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:27.344321 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1266/2552] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 22:02:27 2022] PUT /identity/v3/projects/215b29e74d4e41118d1a0d11742d849b/users/6101d4540a3e44a197b0c007ec8c61e5/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:27.354548 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3527bc-6a3d-4be2-8079-72d7b4db3634 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:27.355038 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3527bc-6a3d-4be2-8079-72d7b4db3634 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:27.355151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3527bc-6a3d-4be2-8079-72d7b4db3634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:27.382156 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-8e3527bc-6a3d-4be2-8079-72d7b4db3634 None None] MFA Rules not processed for user `6101d4540a3e44a197b0c007ec8c61e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:27.383390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-8e3527bc-6a3d-4be2-8079-72d7b4db3634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:27.448951 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1287/2553] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:27.460101 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d1846-18f2-4689-9719-aa0635c42463 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:27.461876 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-4c1d1846-18f2-4689-9719-aa0635c42463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:27.519859 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d1846-18f2-4689-9719-aa0635c42463 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:27.523454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd38c2e-d49c-4103-9b64-0bda9880885f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:27.524076 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd38c2e-d49c-4103-9b64-0bda9880885f service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:27.524355 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd38c2e-d49c-4103-9b64-0bda9880885f service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:27.524628 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd38c2e-d49c-4103-9b64-0bda9880885f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:27.534028 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd38c2e-d49c-4103-9b64-0bda9880885f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:27.535431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd38c2e-d49c-4103-9b64-0bda9880885f service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:27.544847 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1267/2554] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:02:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:02:27.573487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3527bc-6a3d-4be2-8079-72d7b4db3634 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:27.634007 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3527bc-6a3d-4be2-8079-72d7b4db3634 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:27.639224 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa064d-f807-4130-aa5e-c1f60e7cb605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:27.639913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa064d-f807-4130-aa5e-c1f60e7cb605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:27.640193 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa064d-f807-4130-aa5e-c1f60e7cb605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:27.640467 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa064d-f807-4130-aa5e-c1f60e7cb605 admin admin] PATH_INFO: `/v3/users/8b49ef30c2a348018bdbcba5ba13ef56` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:27.641601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa064d-f807-4130-aa5e-c1f60e7cb605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b49ef30c2a348018bdbcba5ba13ef56)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:27.643587 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa064d-f807-4130-aa5e-c1f60e7cb605 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:27.685695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:27.692608 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b9fa064d-f807-4130-aa5e-c1f60e7cb605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b49ef30c2a348018bdbcba5ba13ef56'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:27.705807 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b9fa064d-f807-4130-aa5e-c1f60e7cb605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b49ef30c2a348018bdbcba5ba13ef56'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:27.713097 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b9fa064d-f807-4130-aa5e-c1f60e7cb605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b49ef30c2a348018bdbcba5ba13ef56'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:27.713778 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b9fa064d-f807-4130-aa5e-c1f60e7cb605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b49ef30c2a348018bdbcba5ba13ef56'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:27.720621 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b9fa064d-f807-4130-aa5e-c1f60e7cb605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b49ef30c2a348018bdbcba5ba13ef56'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:27.726977 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1288/2555] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 22:02:27 2022] DELETE /identity/v3/users/8b49ef30c2a348018bdbcba5ba13ef56 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:27.731518 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b3d73-c2a8-48c8-bfb1-29f5ae753b3b None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:27.731839 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b3d73-c2a8-48c8-bfb1-29f5ae753b3b None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:27.732115 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b3d73-c2a8-48c8-bfb1-29f5ae753b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:27.751807 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-806b3d73-c2a8-48c8-bfb1-29f5ae753b3b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:27.753004 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-806b3d73-c2a8-48c8-bfb1-29f5ae753b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:27.814011 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1268/2556] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:27.824347 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa064d-f807-4130-aa5e-c1f60e7cb605 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:27.825766 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b9fa064d-f807-4130-aa5e-c1f60e7cb605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:27.882827 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa064d-f807-4130-aa5e-c1f60e7cb605 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:27.887818 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d35ad-f7fe-4a95-abd9-697a9880ee26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:27.888423 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d35ad-f7fe-4a95-abd9-697a9880ee26 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:27.888707 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d35ad-f7fe-4a95-abd9-697a9880ee26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:27.888979 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d35ad-f7fe-4a95-abd9-697a9880ee26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:27.898242 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d35ad-f7fe-4a95-abd9-697a9880ee26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:27.899710 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d35ad-f7fe-4a95-abd9-697a9880ee26 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:27.909831 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1289/2557] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:02:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:02:28.312103 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b3d73-c2a8-48c8-bfb1-29f5ae753b3b None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:28.313638 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-806b3d73-c2a8-48c8-bfb1-29f5ae753b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:28.326264 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b3d73-c2a8-48c8-bfb1-29f5ae753b3b None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:28.331400 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff65d16-0d24-427f-93e2-a15ad05036b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:28.332098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff65d16-0d24-427f-93e2-a15ad05036b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:28.332382 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff65d16-0d24-427f-93e2-a15ad05036b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:28.332654 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff65d16-0d24-427f-93e2-a15ad05036b6 admin admin] PATH_INFO: `/v3/projects/b69e545b13eb4c599f21591f628b7ecd` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:28.333810 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff65d16-0d24-427f-93e2-a15ad05036b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b69e545b13eb4c599f21591f628b7ecd)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:28.335796 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff65d16-0d24-427f-93e2-a15ad05036b6 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:28.386811 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9ff65d16-0d24-427f-93e2-a15ad05036b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b69e545b13eb4c599f21591f628b7ecd', 'request_id': 'req-9ff65d16-0d24-427f-93e2-a15ad05036b6'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:28.395012 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9ff65d16-0d24-427f-93e2-a15ad05036b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b69e545b13eb4c599f21591f628b7ecd', 'request_id': 'req-9ff65d16-0d24-427f-93e2-a15ad05036b6'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:28.402050 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9ff65d16-0d24-427f-93e2-a15ad05036b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b69e545b13eb4c599f21591f628b7ecd', 'request_id': 'req-9ff65d16-0d24-427f-93e2-a15ad05036b6'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:28.414250 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9ff65d16-0d24-427f-93e2-a15ad05036b6 admin admin] The token cache is being invalidate because project b69e545b13eb4c599f21591f628b7ecd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 22:02:28.414670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9ff65d16-0d24-427f-93e2-a15ad05036b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:28.415289 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-9ff65d16-0d24-427f-93e2-a15ad05036b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:28.417406 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1269/2558] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 22:02:28 2022] DELETE /identity/v3/projects/b69e545b13eb4c599f21591f628b7ecd => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:28.422142 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d35ad-f7fe-4a95-abd9-697a9880ee26 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:28.432787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d35ad-f7fe-4a95-abd9-697a9880ee26 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:28.488010 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6944e6c-991a-497e-8ed2-65ff4d49ca9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:28.488633 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6944e6c-991a-497e-8ed2-65ff4d49ca9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:28.488912 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6944e6c-991a-497e-8ed2-65ff4d49ca9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:28.489181 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6944e6c-991a-497e-8ed2-65ff4d49ca9c admin admin] PATH_INFO: `/v3/users/6101d4540a3e44a197b0c007ec8c61e5` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:28.490276 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6944e6c-991a-497e-8ed2-65ff4d49ca9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6101d4540a3e44a197b0c007ec8c61e5)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:28.492222 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6944e6c-991a-497e-8ed2-65ff4d49ca9c admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:28.536382 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b6944e6c-991a-497e-8ed2-65ff4d49ca9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6101d4540a3e44a197b0c007ec8c61e5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:28.547208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:28.549687 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b6944e6c-991a-497e-8ed2-65ff4d49ca9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6101d4540a3e44a197b0c007ec8c61e5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:28.557441 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b6944e6c-991a-497e-8ed2-65ff4d49ca9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6101d4540a3e44a197b0c007ec8c61e5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:28.557985 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b6944e6c-991a-497e-8ed2-65ff4d49ca9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6101d4540a3e44a197b0c007ec8c61e5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:28.564896 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-b6944e6c-991a-497e-8ed2-65ff4d49ca9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6101d4540a3e44a197b0c007ec8c61e5'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:28.570712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1290/2559] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 22:02:28 2022] DELETE /identity/v3/users/6101d4540a3e44a197b0c007ec8c61e5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:28.838264 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff65d16-0d24-427f-93e2-a15ad05036b6 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:28.840472 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9ff65d16-0d24-427f-93e2-a15ad05036b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:28.853373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff65d16-0d24-427f-93e2-a15ad05036b6 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:28.865577 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:28.911555 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93faf93c-edda-48c9-952f-72c3d0ec1f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:28.912200 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93faf93c-edda-48c9-952f-72c3d0ec1f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:28.912476 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93faf93c-edda-48c9-952f-72c3d0ec1f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:28.912745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93faf93c-edda-48c9-952f-72c3d0ec1f40 admin admin] PATH_INFO: `/v3/projects/215b29e74d4e41118d1a0d11742d849b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:28.913837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93faf93c-edda-48c9-952f-72c3d0ec1f40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=215b29e74d4e41118d1a0d11742d849b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:28.915736 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93faf93c-edda-48c9-952f-72c3d0ec1f40 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:28.959741 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-93faf93c-edda-48c9-952f-72c3d0ec1f40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '215b29e74d4e41118d1a0d11742d849b', 'request_id': 'req-93faf93c-edda-48c9-952f-72c3d0ec1f40'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:28.967096 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-93faf93c-edda-48c9-952f-72c3d0ec1f40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '215b29e74d4e41118d1a0d11742d849b', 'request_id': 'req-93faf93c-edda-48c9-952f-72c3d0ec1f40'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:28.972545 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-93faf93c-edda-48c9-952f-72c3d0ec1f40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '215b29e74d4e41118d1a0d11742d849b', 'request_id': 'req-93faf93c-edda-48c9-952f-72c3d0ec1f40'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:28.983351 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-93faf93c-edda-48c9-952f-72c3d0ec1f40 admin admin] The token cache is being invalidate because project 215b29e74d4e41118d1a0d11742d849b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 22:02:28.983663 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-93faf93c-edda-48c9-952f-72c3d0ec1f40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:28.984283 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-93faf93c-edda-48c9-952f-72c3d0ec1f40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:28.985728 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1270/2560] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 22:02:28 2022] DELETE /identity/v3/projects/215b29e74d4e41118d1a0d11742d849b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:28.990695 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bfcb9-41c0-499a-8fe8-235f8a5593e8 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:28.990983 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bfcb9-41c0-499a-8fe8-235f8a5593e8 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:28.991281 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bfcb9-41c0-499a-8fe8-235f8a5593e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:29.011380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-434bfcb9-41c0-499a-8fe8-235f8a5593e8 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:29.012630 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-434bfcb9-41c0-499a-8fe8-235f8a5593e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:29.069771 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1291/2561] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:29.078020 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99ecc8-bee3-46c7-baed-5921441c904a None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:29.078301 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99ecc8-bee3-46c7-baed-5921441c904a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:29.078572 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99ecc8-bee3-46c7-baed-5921441c904a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:29.097971 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-7a99ecc8-bee3-46c7-baed-5921441c904a None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:29.099161 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-7a99ecc8-bee3-46c7-baed-5921441c904a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:29.106934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1271/2562] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:29.111025 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434bfcb9-41c0-499a-8fe8-235f8a5593e8 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:29.121208 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434bfcb9-41c0-499a-8fe8-235f8a5593e8 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:29.125684 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e9f2b-11eb-45eb-be83-760bac5db01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:29.126294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e9f2b-11eb-45eb-be83-760bac5db01e admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:29.126557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e9f2b-11eb-45eb-be83-760bac5db01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:29.126831 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e9f2b-11eb-45eb-be83-760bac5db01e admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:29.127461 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e9f2b-11eb-45eb-be83-760bac5db01e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 22:02:29.127801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e9f2b-11eb-45eb-be83-760bac5db01e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:29.129390 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e9f2b-11eb-45eb-be83-760bac5db01e admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:29.141314 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1292/2563] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 22:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 22:02:29.145414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99ecc8-bee3-46c7-baed-5921441c904a None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:29.154518 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99ecc8-bee3-46c7-baed-5921441c904a None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:29.159084 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca48a2-5950-48fe-b0e9-0829573dd1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:29.159704 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca48a2-5950-48fe-b0e9-0829573dd1bf admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:29.159987 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca48a2-5950-48fe-b0e9-0829573dd1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:29.160259 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca48a2-5950-48fe-b0e9-0829573dd1bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:29.160878 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca48a2-5950-48fe-b0e9-0829573dd1bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:29.162744 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca48a2-5950-48fe-b0e9-0829573dd1bf admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:29.189203 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1272/2564] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 22:02:29 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:29.193253 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e9f2b-11eb-45eb-be83-760bac5db01e admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:29.203577 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e9f2b-11eb-45eb-be83-760bac5db01e admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:29.257649 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576d5f2-e710-48bd-bc7e-accc4924258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:29.258276 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576d5f2-e710-48bd-bc7e-accc4924258b admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:29.258557 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576d5f2-e710-48bd-bc7e-accc4924258b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:29.258832 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576d5f2-e710-48bd-bc7e-accc4924258b admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:29.259453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576d5f2-e710-48bd-bc7e-accc4924258b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:29.262008 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576d5f2-e710-48bd-bc7e-accc4924258b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:29.283169 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1293/2565] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 22:02:29 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:29.287338 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca48a2-5950-48fe-b0e9-0829573dd1bf admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:29.297624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca48a2-5950-48fe-b0e9-0829573dd1bf admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:29.302280 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120108b-84f3-46b9-8342-ba0db90ad263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:29.302877 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120108b-84f3-46b9-8342-ba0db90ad263 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:29.303159 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120108b-84f3-46b9-8342-ba0db90ad263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:29.303434 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120108b-84f3-46b9-8342-ba0db90ad263 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:29.304123 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120108b-84f3-46b9-8342-ba0db90ad263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:29.305700 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120108b-84f3-46b9-8342-ba0db90ad263 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:29.315968 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1273/2566] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:29.319649 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576d5f2-e710-48bd-bc7e-accc4924258b admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:29.330027 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576d5f2-e710-48bd-bc7e-accc4924258b admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:29.334599 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05458b02-2014-4ceb-a509-db7d70860759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:29.335185 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05458b02-2014-4ceb-a509-db7d70860759 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:29.335462 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05458b02-2014-4ceb-a509-db7d70860759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:29.335766 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05458b02-2014-4ceb-a509-db7d70860759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:29.336389 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05458b02-2014-4ceb-a509-db7d70860759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:29.337973 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05458b02-2014-4ceb-a509-db7d70860759 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:29.348313 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1294/2567] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:29.352267 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120108b-84f3-46b9-8342-ba0db90ad263 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:29.362514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120108b-84f3-46b9-8342-ba0db90ad263 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:29.367194 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ff2af-549b-4553-931c-1d7551288a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:29.367869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ff2af-549b-4553-931c-1d7551288a5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:29.368149 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ff2af-549b-4553-931c-1d7551288a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:29.368419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ff2af-549b-4553-931c-1d7551288a5d admin admin] PATH_INFO: `/v3/projects/facc7fe72bed4d908a251dc5089dce6b/users/1c4262435df44cda85c8f1719e58af92/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:29.383445 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ff2af-549b-4553-931c-1d7551288a5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=facc7fe72bed4d908a251dc5089dce6b, user_id=1c4262435df44cda85c8f1719e58af92, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:29.386753 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ff2af-549b-4553-931c-1d7551288a5d admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:29.399302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1274/2568] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 22:02:29 2022] PUT /identity/v3/projects/facc7fe72bed4d908a251dc5089dce6b/users/1c4262435df44cda85c8f1719e58af92/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:29.410590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da89d4-de9c-4b90-8b41-3693f229113b None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:29.410881 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da89d4-de9c-4b90-8b41-3693f229113b None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:29.411153 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da89d4-de9c-4b90-8b41-3693f229113b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:29.439303 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-d1da89d4-de9c-4b90-8b41-3693f229113b None None] MFA Rules not processed for user `1c4262435df44cda85c8f1719e58af92`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:29.440550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-d1da89d4-de9c-4b90-8b41-3693f229113b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:29.471913 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:29.505552 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1295/2569] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:29.510347 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ff2af-549b-4553-931c-1d7551288a5d admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:29.553454 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:29.571940 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ff2af-549b-4553-931c-1d7551288a5d admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:29.576540 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a833c2-d0e2-4b45-8642-ad9a9971ec29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:29.577133 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a833c2-d0e2-4b45-8642-ad9a9971ec29 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:29.577414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a833c2-d0e2-4b45-8642-ad9a9971ec29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:29.577686 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a833c2-d0e2-4b45-8642-ad9a9971ec29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:29.578304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a833c2-d0e2-4b45-8642-ad9a9971ec29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:29.580329 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a833c2-d0e2-4b45-8642-ad9a9971ec29 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:29.606023 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1275/2570] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 22:02:29 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:29.610079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da89d4-de9c-4b90-8b41-3693f229113b None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:29.621157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da89d4-de9c-4b90-8b41-3693f229113b None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:29.676012 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371a340-9858-4eef-8424-7a52817c391f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:29.676605 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371a340-9858-4eef-8424-7a52817c391f admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:29.676883 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371a340-9858-4eef-8424-7a52817c391f admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:29.677154 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371a340-9858-4eef-8424-7a52817c391f admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:29.677764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d371a340-9858-4eef-8424-7a52817c391f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:29.679702 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d371a340-9858-4eef-8424-7a52817c391f admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:29.700547 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1296/2571] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 22:02:29 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:29.704486 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a833c2-d0e2-4b45-8642-ad9a9971ec29 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:29.714578 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a833c2-d0e2-4b45-8642-ad9a9971ec29 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:29.719114 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c04a1-daa2-4b4a-a565-43aa78a84dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:29.719742 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c04a1-daa2-4b4a-a565-43aa78a84dfe admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:29.720024 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c04a1-daa2-4b4a-a565-43aa78a84dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:29.720299 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c04a1-daa2-4b4a-a565-43aa78a84dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:29.720935 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c04a1-daa2-4b4a-a565-43aa78a84dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:29.726763 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c04a1-daa2-4b4a-a565-43aa78a84dfe admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:29.734429 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1276/2572] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:29.738560 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371a340-9858-4eef-8424-7a52817c391f admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:29.749152 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371a340-9858-4eef-8424-7a52817c391f admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:29.753687 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6a76f-093f-4dd2-8b05-4d8a8faf36c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:29.754323 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6a76f-093f-4dd2-8b05-4d8a8faf36c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:29.754597 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6a76f-093f-4dd2-8b05-4d8a8faf36c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:29.754866 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6a76f-093f-4dd2-8b05-4d8a8faf36c8 admin admin] PATH_INFO: `/v3/projects/ea72479598254f8fad80f3a3a114cca6/users/8a55bf75a42d437e8e3d7d1b577ccc3b/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:29.769118 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6a76f-093f-4dd2-8b05-4d8a8faf36c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea72479598254f8fad80f3a3a114cca6, user_id=8a55bf75a42d437e8e3d7d1b577ccc3b, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:29.771784 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6a76f-093f-4dd2-8b05-4d8a8faf36c8 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:29.785142 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1297/2573] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 22:02:29 2022] PUT /identity/v3/projects/ea72479598254f8fad80f3a3a114cca6/users/8a55bf75a42d437e8e3d7d1b577ccc3b/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 22:02:29.789170 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c04a1-daa2-4b4a-a565-43aa78a84dfe admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:29.799494 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c04a1-daa2-4b4a-a565-43aa78a84dfe admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:29.855206 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd716f-1c46-43d9-9116-df7d1edac1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:29.855869 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd716f-1c46-43d9-9116-df7d1edac1a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:29.855965 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd716f-1c46-43d9-9116-df7d1edac1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:29.856241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd716f-1c46-43d9-9116-df7d1edac1a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:29.857098 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd716f-1c46-43d9-9116-df7d1edac1a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:29.858786 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd716f-1c46-43d9-9116-df7d1edac1a4 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:29.868522 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1277/2574] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:29.872675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6a76f-093f-4dd2-8b05-4d8a8faf36c8 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:29.883089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6a76f-093f-4dd2-8b05-4d8a8faf36c8 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:29.887705 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273467ce-68fc-44e9-b6a4-d231e52239a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:29.888353 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273467ce-68fc-44e9-b6a4-d231e52239a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:29.888632 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273467ce-68fc-44e9-b6a4-d231e52239a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:29.888902 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273467ce-68fc-44e9-b6a4-d231e52239a8 admin admin] PATH_INFO: `/v3/projects/ea72479598254f8fad80f3a3a114cca6/users/8a55bf75a42d437e8e3d7d1b577ccc3b/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:29.890872 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273467ce-68fc-44e9-b6a4-d231e52239a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea72479598254f8fad80f3a3a114cca6, user_id=8a55bf75a42d437e8e3d7d1b577ccc3b, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:29.893628 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273467ce-68fc-44e9-b6a4-d231e52239a8 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:29.904176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1298/2575] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 22:02:29 2022] PUT /identity/v3/projects/ea72479598254f8fad80f3a3a114cca6/users/8a55bf75a42d437e8e3d7d1b577ccc3b/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:29.914019 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cb6e3-62ff-4fc1-ba33-390a95e87784 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:29.914304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cb6e3-62ff-4fc1-ba33-390a95e87784 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:29.914606 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cb6e3-62ff-4fc1-ba33-390a95e87784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:29.942113 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-723cb6e3-62ff-4fc1-ba33-390a95e87784 None None] MFA Rules not processed for user `8a55bf75a42d437e8e3d7d1b577ccc3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:29.943311 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-723cb6e3-62ff-4fc1-ba33-390a95e87784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:30.010056 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1278/2576] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:30.020905 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273467ce-68fc-44e9-b6a4-d231e52239a8 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:30.022575 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-273467ce-68fc-44e9-b6a4-d231e52239a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:30.078786 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273467ce-68fc-44e9-b6a4-d231e52239a8 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:30.082385 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e45da9-06ec-4893-90db-13b18fc924af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:30.082976 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e45da9-06ec-4893-90db-13b18fc924af service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:30.083252 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e45da9-06ec-4893-90db-13b18fc924af service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:30.083524 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e45da9-06ec-4893-90db-13b18fc924af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:30.093004 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e45da9-06ec-4893-90db-13b18fc924af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:30.094423 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e45da9-06ec-4893-90db-13b18fc924af service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:30.103780 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1299/2577] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:02:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:02:30.671741 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cb6e3-62ff-4fc1-ba33-390a95e87784 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:30.713801 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:30.736655 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cb6e3-62ff-4fc1-ba33-390a95e87784 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:30.741127 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80682ff4-a480-439d-80e6-45d0ce975fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:30.741753 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80682ff4-a480-439d-80e6-45d0ce975fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:30.742033 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80682ff4-a480-439d-80e6-45d0ce975fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:30.742308 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80682ff4-a480-439d-80e6-45d0ce975fe1 admin admin] PATH_INFO: `/v3/users/1c4262435df44cda85c8f1719e58af92` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:30.743416 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80682ff4-a480-439d-80e6-45d0ce975fe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c4262435df44cda85c8f1719e58af92)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:30.745394 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80682ff4-a480-439d-80e6-45d0ce975fe1 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:30.789939 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-80682ff4-a480-439d-80e6-45d0ce975fe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c4262435df44cda85c8f1719e58af92'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:30.790585 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-80682ff4-a480-439d-80e6-45d0ce975fe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c4262435df44cda85c8f1719e58af92'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:30.802670 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-80682ff4-a480-439d-80e6-45d0ce975fe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c4262435df44cda85c8f1719e58af92'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:30.809698 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-80682ff4-a480-439d-80e6-45d0ce975fe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c4262435df44cda85c8f1719e58af92'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:30.810321 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-80682ff4-a480-439d-80e6-45d0ce975fe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c4262435df44cda85c8f1719e58af92'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:30.822134 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1279/2578] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 22:02:30 2022] DELETE /identity/v3/users/1c4262435df44cda85c8f1719e58af92 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:30.826686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c76d9-56e0-47a5-ba16-ae244d98cf8b None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:30.827157 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c76d9-56e0-47a5-ba16-ae244d98cf8b None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:30.827248 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c76d9-56e0-47a5-ba16-ae244d98cf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:30.845875 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-f20c76d9-56e0-47a5-ba16-ae244d98cf8b None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:30.847060 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f20c76d9-56e0-47a5-ba16-ae244d98cf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:30.870548 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:30.905477 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1300/2579] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:30.914932 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80682ff4-a480-439d-80e6-45d0ce975fe1 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:30.916318 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-80682ff4-a480-439d-80e6-45d0ce975fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:30.972526 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80682ff4-a480-439d-80e6-45d0ce975fe1 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:30.976264 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a661de-96da-45b8-a34c-ed8add783197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:30.976866 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a661de-96da-45b8-a34c-ed8add783197 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:30.977146 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a661de-96da-45b8-a34c-ed8add783197 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:30.977417 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a661de-96da-45b8-a34c-ed8add783197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:30.986245 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a661de-96da-45b8-a34c-ed8add783197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:30.987699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a661de-96da-45b8-a34c-ed8add783197 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:30.997038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1280/2580] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:02:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:02:31.253642 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c76d9-56e0-47a5-ba16-ae244d98cf8b None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:31.255378 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f20c76d9-56e0-47a5-ba16-ae244d98cf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:31.270548 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c76d9-56e0-47a5-ba16-ae244d98cf8b None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:31.275877 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67fa68-948d-4388-b684-cd02b7921809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:31.276677 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67fa68-948d-4388-b684-cd02b7921809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:31.277104 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67fa68-948d-4388-b684-cd02b7921809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:31.277513 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67fa68-948d-4388-b684-cd02b7921809 admin admin] PATH_INFO: `/v3/projects/facc7fe72bed4d908a251dc5089dce6b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:31.278854 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67fa68-948d-4388-b684-cd02b7921809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=facc7fe72bed4d908a251dc5089dce6b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:31.281285 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67fa68-948d-4388-b684-cd02b7921809 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:31.327708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-6d67fa68-948d-4388-b684-cd02b7921809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'facc7fe72bed4d908a251dc5089dce6b', 'request_id': 'req-6d67fa68-948d-4388-b684-cd02b7921809'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:31.334182 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-6d67fa68-948d-4388-b684-cd02b7921809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'facc7fe72bed4d908a251dc5089dce6b', 'request_id': 'req-6d67fa68-948d-4388-b684-cd02b7921809'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:31.341599 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-6d67fa68-948d-4388-b684-cd02b7921809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'facc7fe72bed4d908a251dc5089dce6b', 'request_id': 'req-6d67fa68-948d-4388-b684-cd02b7921809'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:31.351793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-6d67fa68-948d-4388-b684-cd02b7921809 admin admin] The token cache is being invalidate because project facc7fe72bed4d908a251dc5089dce6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 22:02:31.352079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-6d67fa68-948d-4388-b684-cd02b7921809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:31.352726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-6d67fa68-948d-4388-b684-cd02b7921809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:31.354201 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1301/2581] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 22:02:31 2022] DELETE /identity/v3/projects/facc7fe72bed4d908a251dc5089dce6b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:31.358251 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a661de-96da-45b8-a34c-ed8add783197 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:31.367817 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a661de-96da-45b8-a34c-ed8add783197 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:31.423320 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9deb70-b512-4428-9378-fcdb4d991b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:31.423984 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9deb70-b512-4428-9378-fcdb4d991b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:31.424262 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9deb70-b512-4428-9378-fcdb4d991b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:31.424530 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9deb70-b512-4428-9378-fcdb4d991b0c admin admin] PATH_INFO: `/v3/users/8a55bf75a42d437e8e3d7d1b577ccc3b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:31.425635 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9deb70-b512-4428-9378-fcdb4d991b0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a55bf75a42d437e8e3d7d1b577ccc3b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:31.427525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9deb70-b512-4428-9378-fcdb4d991b0c admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:31.429049 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:31.473833 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-7b9deb70-b512-4428-9378-fcdb4d991b0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a55bf75a42d437e8e3d7d1b577ccc3b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:31.474475 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-7b9deb70-b512-4428-9378-fcdb4d991b0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a55bf75a42d437e8e3d7d1b577ccc3b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:31.485896 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-7b9deb70-b512-4428-9378-fcdb4d991b0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a55bf75a42d437e8e3d7d1b577ccc3b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:31.493021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-7b9deb70-b512-4428-9378-fcdb4d991b0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a55bf75a42d437e8e3d7d1b577ccc3b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:31.493630 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-7b9deb70-b512-4428-9378-fcdb4d991b0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a55bf75a42d437e8e3d7d1b577ccc3b'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:31.506499 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1281/2582] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 22:02:31 2022] DELETE /identity/v3/users/8a55bf75a42d437e8e3d7d1b577ccc3b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:31.784961 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67fa68-948d-4388-b684-cd02b7921809 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:31.786389 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-6d67fa68-948d-4388-b684-cd02b7921809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:31.800003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67fa68-948d-4388-b684-cd02b7921809 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:31.855244 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd76304-1165-4627-96c2-98d92dd4bf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:31.855911 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd76304-1165-4627-96c2-98d92dd4bf32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:31.856665 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd76304-1165-4627-96c2-98d92dd4bf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:31.856932 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd76304-1165-4627-96c2-98d92dd4bf32 admin admin] PATH_INFO: `/v3/projects/ea72479598254f8fad80f3a3a114cca6` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:31.858020 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd76304-1165-4627-96c2-98d92dd4bf32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea72479598254f8fad80f3a3a114cca6)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:31.859940 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd76304-1165-4627-96c2-98d92dd4bf32 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:31.905738 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4cd76304-1165-4627-96c2-98d92dd4bf32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea72479598254f8fad80f3a3a114cca6', 'request_id': 'req-4cd76304-1165-4627-96c2-98d92dd4bf32'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:31.911772 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4cd76304-1165-4627-96c2-98d92dd4bf32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea72479598254f8fad80f3a3a114cca6', 'request_id': 'req-4cd76304-1165-4627-96c2-98d92dd4bf32'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:31.913403 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:31.920114 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4cd76304-1165-4627-96c2-98d92dd4bf32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea72479598254f8fad80f3a3a114cca6', 'request_id': 'req-4cd76304-1165-4627-96c2-98d92dd4bf32'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:31.931194 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4cd76304-1165-4627-96c2-98d92dd4bf32 admin admin] The token cache is being invalidate because project ea72479598254f8fad80f3a3a114cca6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 22:02:31.931491 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4cd76304-1165-4627-96c2-98d92dd4bf32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:31.932168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-4cd76304-1165-4627-96c2-98d92dd4bf32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:31.933718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1302/2583] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 22:02:31 2022] DELETE /identity/v3/projects/ea72479598254f8fad80f3a3a114cca6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:31.939216 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339492c-627f-41b1-b624-3c7925490f19 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:31.939699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339492c-627f-41b1-b624-3c7925490f19 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:31.939813 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339492c-627f-41b1-b624-3c7925490f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:31.959896 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-e339492c-627f-41b1-b624-3c7925490f19 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:31.961032 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-e339492c-627f-41b1-b624-3c7925490f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:32.018302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1282/2584] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:32.026946 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3091589-6f7e-4b1a-bf53-f367b443e7b2 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:32.027231 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3091589-6f7e-4b1a-bf53-f367b443e7b2 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:32.027500 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3091589-6f7e-4b1a-bf53-f367b443e7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:32.047712 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-b3091589-6f7e-4b1a-bf53-f367b443e7b2 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:32.048887 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-b3091589-6f7e-4b1a-bf53-f367b443e7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:32.056533 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1303/2585] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:32.060478 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339492c-627f-41b1-b624-3c7925490f19 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:32.069991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339492c-627f-41b1-b624-3c7925490f19 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:32.074515 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e77233-9568-4add-a1cf-acd2d2bb8f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:32.075136 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e77233-9568-4add-a1cf-acd2d2bb8f28 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:32.075414 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e77233-9568-4add-a1cf-acd2d2bb8f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:32.075729 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e77233-9568-4add-a1cf-acd2d2bb8f28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:32.076369 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e77233-9568-4add-a1cf-acd2d2bb8f28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 22:02:32.076699 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e77233-9568-4add-a1cf-acd2d2bb8f28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:32.078268 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e77233-9568-4add-a1cf-acd2d2bb8f28 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:32.089584 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1283/2586] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 22:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 22:02:32.094110 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3091589-6f7e-4b1a-bf53-f367b443e7b2 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:32.103058 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3091589-6f7e-4b1a-bf53-f367b443e7b2 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:32.108226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3c473-127b-4b26-a1ce-d7dff8c2ec4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:32.108812 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3c473-127b-4b26-a1ce-d7dff8c2ec4b admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:32.109085 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3c473-127b-4b26-a1ce-d7dff8c2ec4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:32.109353 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3c473-127b-4b26-a1ce-d7dff8c2ec4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:32.109970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3c473-127b-4b26-a1ce-d7dff8c2ec4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:32.111890 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3c473-127b-4b26-a1ce-d7dff8c2ec4b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:32.138090 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1304/2587] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 22:02:32 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:32.142045 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e77233-9568-4add-a1cf-acd2d2bb8f28 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:32.152022 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e77233-9568-4add-a1cf-acd2d2bb8f28 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:32.206247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e107265f-d32a-49d7-a35a-864045aebd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:32.206839 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107265f-d32a-49d7-a35a-864045aebd02 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:32.207116 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107265f-d32a-49d7-a35a-864045aebd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:32.207386 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107265f-d32a-49d7-a35a-864045aebd02 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:32.208058 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e107265f-d32a-49d7-a35a-864045aebd02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:32.209994 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e107265f-d32a-49d7-a35a-864045aebd02 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:32.221048 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:32.233125 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1284/2588] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 22:02:32 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:32.237452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3c473-127b-4b26-a1ce-d7dff8c2ec4b admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:32.247412 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3c473-127b-4b26-a1ce-d7dff8c2ec4b admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:32.251964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd530c-70be-4828-9e8d-e4eea6231e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:32.252549 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd530c-70be-4828-9e8d-e4eea6231e02 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:32.252824 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd530c-70be-4828-9e8d-e4eea6231e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:32.253089 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd530c-70be-4828-9e8d-e4eea6231e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:32.253713 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd530c-70be-4828-9e8d-e4eea6231e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:32.255296 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd530c-70be-4828-9e8d-e4eea6231e02 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:32.265292 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1305/2589] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:32.269228 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e107265f-d32a-49d7-a35a-864045aebd02 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:32.279464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e107265f-d32a-49d7-a35a-864045aebd02 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:32.284013 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db0b41-0295-412f-9796-46f077de8b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:32.284627 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db0b41-0295-412f-9796-46f077de8b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:32.284906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db0b41-0295-412f-9796-46f077de8b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:32.285196 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db0b41-0295-412f-9796-46f077de8b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:32.285798 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db0b41-0295-412f-9796-46f077de8b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:32.287373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db0b41-0295-412f-9796-46f077de8b3b admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:32.298531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1285/2590] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:32.302606 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd530c-70be-4828-9e8d-e4eea6231e02 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:32.312475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd530c-70be-4828-9e8d-e4eea6231e02 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:32.317270 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe8f6b-dee9-4503-b2fc-3134a573370d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:32.317948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe8f6b-dee9-4503-b2fc-3134a573370d admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:32.318288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe8f6b-dee9-4503-b2fc-3134a573370d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:32.318590 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe8f6b-dee9-4503-b2fc-3134a573370d admin admin] PATH_INFO: `/v3/projects/ac3de222498146c1a0cde366b9c6c7ee/users/d0f65437ca934fe3a60e6bce33e5e87b/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:32.332967 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe8f6b-dee9-4503-b2fc-3134a573370d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac3de222498146c1a0cde366b9c6c7ee, user_id=d0f65437ca934fe3a60e6bce33e5e87b, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:32.335645 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe8f6b-dee9-4503-b2fc-3134a573370d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:32.349511 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1306/2591] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 22:02:32 2022] PUT /identity/v3/projects/ac3de222498146c1a0cde366b9c6c7ee/users/d0f65437ca934fe3a60e6bce33e5e87b/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:32.361343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc86ba-40de-44ec-9d2c-2ce0b50a648d None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:32.361631 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc86ba-40de-44ec-9d2c-2ce0b50a648d None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:32.361903 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc86ba-40de-44ec-9d2c-2ce0b50a648d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:32.390842 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-43dc86ba-40de-44ec-9d2c-2ce0b50a648d None None] MFA Rules not processed for user `d0f65437ca934fe3a60e6bce33e5e87b`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:32.392078 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-43dc86ba-40de-44ec-9d2c-2ce0b50a648d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:32.456080 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1286/2592] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:32.460705 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe8f6b-dee9-4503-b2fc-3134a573370d admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:32.520009 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe8f6b-dee9-4503-b2fc-3134a573370d admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:32.524689 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a637b0-9f78-4e91-961a-ff9f10e19306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:32.525289 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a637b0-9f78-4e91-961a-ff9f10e19306 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:32.525571 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a637b0-9f78-4e91-961a-ff9f10e19306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:32.525843 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a637b0-9f78-4e91-961a-ff9f10e19306 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:32.526473 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a637b0-9f78-4e91-961a-ff9f10e19306 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:32.528964 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a637b0-9f78-4e91-961a-ff9f10e19306 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:32.554458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1307/2593] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 22:02:32 2022] POST /identity/v3/projects => generated 381 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:32.558708 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc86ba-40de-44ec-9d2c-2ce0b50a648d None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:32.569917 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc86ba-40de-44ec-9d2c-2ce0b50a648d None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:32.625996 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862ada2-2b14-48a4-804e-460140931920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:32.626600 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862ada2-2b14-48a4-804e-460140931920 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:32.626880 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862ada2-2b14-48a4-804e-460140931920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:32.627155 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862ada2-2b14-48a4-804e-460140931920 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:32.627819 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862ada2-2b14-48a4-804e-460140931920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:32.629725 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862ada2-2b14-48a4-804e-460140931920 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:32.650498 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1287/2594] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 22:02:32 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:32.655053 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a637b0-9f78-4e91-961a-ff9f10e19306 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:32.665343 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a637b0-9f78-4e91-961a-ff9f10e19306 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:32.670025 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666382b-eb92-4997-8e5f-bc1e69b7fdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:32.670614 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666382b-eb92-4997-8e5f-bc1e69b7fdfe admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:32.670892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666382b-eb92-4997-8e5f-bc1e69b7fdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:32.671163 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666382b-eb92-4997-8e5f-bc1e69b7fdfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:32.671914 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666382b-eb92-4997-8e5f-bc1e69b7fdfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:32.673511 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666382b-eb92-4997-8e5f-bc1e69b7fdfe admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:32.683821 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1308/2595] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:32.687403 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862ada2-2b14-48a4-804e-460140931920 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:32.697532 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862ada2-2b14-48a4-804e-460140931920 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:32.702637 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685abd62-3eef-46d5-a6b6-7797865e14a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:32.703279 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685abd62-3eef-46d5-a6b6-7797865e14a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:32.703560 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685abd62-3eef-46d5-a6b6-7797865e14a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:32.703984 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685abd62-3eef-46d5-a6b6-7797865e14a2 admin admin] PATH_INFO: `/v3/projects/256c3d7b6e2a41d399d8105f76bacedc/users/9617488f1bb7451297b86c8e7f0bf74f/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:32.719027 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685abd62-3eef-46d5-a6b6-7797865e14a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=256c3d7b6e2a41d399d8105f76bacedc, user_id=9617488f1bb7451297b86c8e7f0bf74f, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:32.721645 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685abd62-3eef-46d5-a6b6-7797865e14a2 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:32.735494 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1288/2596] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 22:02:32 2022] PUT /identity/v3/projects/256c3d7b6e2a41d399d8105f76bacedc/users/9617488f1bb7451297b86c8e7f0bf74f/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:32.739511 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666382b-eb92-4997-8e5f-bc1e69b7fdfe admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:32.750164 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666382b-eb92-4997-8e5f-bc1e69b7fdfe admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:32.796768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:32.806868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5dac5-9846-44be-89e9-20510aabc304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:32.807475 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5dac5-9846-44be-89e9-20510aabc304 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:32.807796 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5dac5-9846-44be-89e9-20510aabc304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:32.808073 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5dac5-9846-44be-89e9-20510aabc304 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:32.808703 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5dac5-9846-44be-89e9-20510aabc304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:32.810836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5dac5-9846-44be-89e9-20510aabc304 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:32.820488 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1309/2597] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:32.824594 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685abd62-3eef-46d5-a6b6-7797865e14a2 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:32.835290 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685abd62-3eef-46d5-a6b6-7797865e14a2 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:32.839849 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9a179-f02f-4ee6-be29-a39ae168c473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:32.840493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9a179-f02f-4ee6-be29-a39ae168c473 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:32.840769 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9a179-f02f-4ee6-be29-a39ae168c473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:32.841044 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9a179-f02f-4ee6-be29-a39ae168c473 admin admin] PATH_INFO: `/v3/projects/256c3d7b6e2a41d399d8105f76bacedc/users/9617488f1bb7451297b86c8e7f0bf74f/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:32.842935 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9a179-f02f-4ee6-be29-a39ae168c473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=256c3d7b6e2a41d399d8105f76bacedc, user_id=9617488f1bb7451297b86c8e7f0bf74f, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:32.845568 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9a179-f02f-4ee6-be29-a39ae168c473 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:32.855480 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1289/2598] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 22:02:32 2022] PUT /identity/v3/projects/256c3d7b6e2a41d399d8105f76bacedc/users/9617488f1bb7451297b86c8e7f0bf74f/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:32.865580 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a927f-6dba-4bfb-a370-7c37ce2d8691 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:32.866047 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a927f-6dba-4bfb-a370-7c37ce2d8691 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:32.866143 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a927f-6dba-4bfb-a370-7c37ce2d8691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:32.894204 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-448a927f-6dba-4bfb-a370-7c37ce2d8691 None None] MFA Rules not processed for user `9617488f1bb7451297b86c8e7f0bf74f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:32.895395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-448a927f-6dba-4bfb-a370-7c37ce2d8691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:32.960959 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1310/2599] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:32.971688 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9a179-f02f-4ee6-be29-a39ae168c473 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:32.973296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-8bd9a179-f02f-4ee6-be29-a39ae168c473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:33.003474 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:33.031988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9a179-f02f-4ee6-be29-a39ae168c473 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:33.035608 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3c76a-a005-4cb3-8263-96f2ea341c8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:33.036215 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3c76a-a005-4cb3-8263-96f2ea341c8d service cinder] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:33.036497 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3c76a-a005-4cb3-8263-96f2ea341c8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:33.036770 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3c76a-a005-4cb3-8263-96f2ea341c8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:33.045572 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3c76a-a005-4cb3-8263-96f2ea341c8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:33.046986 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3c76a-a005-4cb3-8263-96f2ea341c8d service cinder] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:33.056321 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1290/2600] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:02:32 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:02:33.413634 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a927f-6dba-4bfb-a370-7c37ce2d8691 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:33.474946 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a927f-6dba-4bfb-a370-7c37ce2d8691 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:33.479253 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee1210c-1fa2-45f1-85d5-2c4c4ca2d3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:33.479895 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1210c-1fa2-45f1-85d5-2c4c4ca2d3b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:33.480169 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1210c-1fa2-45f1-85d5-2c4c4ca2d3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:33.480434 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1210c-1fa2-45f1-85d5-2c4c4ca2d3b5 admin admin] PATH_INFO: `/v3/users/d0f65437ca934fe3a60e6bce33e5e87b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:33.482113 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee1210c-1fa2-45f1-85d5-2c4c4ca2d3b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0f65437ca934fe3a60e6bce33e5e87b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:33.484046 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee1210c-1fa2-45f1-85d5-2c4c4ca2d3b5 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:33.527033 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-eee1210c-1fa2-45f1-85d5-2c4c4ca2d3b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0f65437ca934fe3a60e6bce33e5e87b'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:33.538885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-eee1210c-1fa2-45f1-85d5-2c4c4ca2d3b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0f65437ca934fe3a60e6bce33e5e87b'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:33.545810 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-eee1210c-1fa2-45f1-85d5-2c4c4ca2d3b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0f65437ca934fe3a60e6bce33e5e87b'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:33.546551 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-eee1210c-1fa2-45f1-85d5-2c4c4ca2d3b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0f65437ca934fe3a60e6bce33e5e87b'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:33.550971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:33.554891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-eee1210c-1fa2-45f1-85d5-2c4c4ca2d3b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0f65437ca934fe3a60e6bce33e5e87b'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:33.561762 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1311/2601] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 22:02:33 2022] DELETE /identity/v3/users/d0f65437ca934fe3a60e6bce33e5e87b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:33.566167 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452456e-04f0-48a5-948a-f07c2db1c821 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:33.566451 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452456e-04f0-48a5-948a-f07c2db1c821 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:33.566723 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452456e-04f0-48a5-948a-f07c2db1c821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:33.585128 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-d452456e-04f0-48a5-948a-f07c2db1c821 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:33.586310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d452456e-04f0-48a5-948a-f07c2db1c821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:33.643385 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1291/2602] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:33.652840 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee1210c-1fa2-45f1-85d5-2c4c4ca2d3b5 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:33.654751 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-eee1210c-1fa2-45f1-85d5-2c4c4ca2d3b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:33.709431 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee1210c-1fa2-45f1-85d5-2c4c4ca2d3b5 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:33.713006 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bd59a-07c1-4387-8ad4-62874d51a319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:33.713601 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bd59a-07c1-4387-8ad4-62874d51a319 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:33.713877 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bd59a-07c1-4387-8ad4-62874d51a319 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:33.714153 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bd59a-07c1-4387-8ad4-62874d51a319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:33.723452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697bd59a-07c1-4387-8ad4-62874d51a319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:33.725194 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697bd59a-07c1-4387-8ad4-62874d51a319 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:33.735760 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1312/2603] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:02:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:02:33.990624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452456e-04f0-48a5-948a-f07c2db1c821 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:33.992121 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-d452456e-04f0-48a5-948a-f07c2db1c821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:34.004162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452456e-04f0-48a5-948a-f07c2db1c821 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:34.009324 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec85e2-4360-4a82-9719-2cc21ac18089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:34.009981 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec85e2-4360-4a82-9719-2cc21ac18089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:34.010255 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec85e2-4360-4a82-9719-2cc21ac18089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:34.010525 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec85e2-4360-4a82-9719-2cc21ac18089 admin admin] PATH_INFO: `/v3/projects/ac3de222498146c1a0cde366b9c6c7ee` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:34.011649 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec85e2-4360-4a82-9719-2cc21ac18089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac3de222498146c1a0cde366b9c6c7ee)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:34.014025 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec85e2-4360-4a82-9719-2cc21ac18089 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:34.059744 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-08ec85e2-4360-4a82-9719-2cc21ac18089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac3de222498146c1a0cde366b9c6c7ee', 'request_id': 'req-08ec85e2-4360-4a82-9719-2cc21ac18089'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:34.067624 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-08ec85e2-4360-4a82-9719-2cc21ac18089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac3de222498146c1a0cde366b9c6c7ee', 'request_id': 'req-08ec85e2-4360-4a82-9719-2cc21ac18089'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:34.073221 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-08ec85e2-4360-4a82-9719-2cc21ac18089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac3de222498146c1a0cde366b9c6c7ee', 'request_id': 'req-08ec85e2-4360-4a82-9719-2cc21ac18089'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:34.084712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-08ec85e2-4360-4a82-9719-2cc21ac18089 admin admin] The token cache is being invalidate because project ac3de222498146c1a0cde366b9c6c7ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 22:02:34.084999 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-08ec85e2-4360-4a82-9719-2cc21ac18089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:34.085625 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-08ec85e2-4360-4a82-9719-2cc21ac18089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:34.087122 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1292/2604] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 22:02:33 2022] DELETE /identity/v3/projects/ac3de222498146c1a0cde366b9c6c7ee => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:34.091725 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bd59a-07c1-4387-8ad4-62874d51a319 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:34.101286 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bd59a-07c1-4387-8ad4-62874d51a319 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:34.155969 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b163ddb-e3aa-4a00-956c-c524a5b3dc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:34.156584 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b163ddb-e3aa-4a00-956c-c524a5b3dc3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:34.156863 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b163ddb-e3aa-4a00-956c-c524a5b3dc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:34.157132 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b163ddb-e3aa-4a00-956c-c524a5b3dc3a admin admin] PATH_INFO: `/v3/users/9617488f1bb7451297b86c8e7f0bf74f` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:34.158226 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b163ddb-e3aa-4a00-956c-c524a5b3dc3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9617488f1bb7451297b86c8e7f0bf74f)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:34.160264 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b163ddb-e3aa-4a00-956c-c524a5b3dc3a admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:34.204646 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9b163ddb-e3aa-4a00-956c-c524a5b3dc3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9617488f1bb7451297b86c8e7f0bf74f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:34.216649 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9b163ddb-e3aa-4a00-956c-c524a5b3dc3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9617488f1bb7451297b86c8e7f0bf74f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:34.223535 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9b163ddb-e3aa-4a00-956c-c524a5b3dc3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9617488f1bb7451297b86c8e7f0bf74f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:34.224178 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9b163ddb-e3aa-4a00-956c-c524a5b3dc3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9617488f1bb7451297b86c8e7f0bf74f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:34.230970 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-9b163ddb-e3aa-4a00-956c-c524a5b3dc3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9617488f1bb7451297b86c8e7f0bf74f'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:34.236829 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1313/2605] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 22:02:34 2022] DELETE /identity/v3/users/9617488f1bb7451297b86c8e7f0bf74f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:34.507350 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec85e2-4360-4a82-9719-2cc21ac18089 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:34.508786 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-08ec85e2-4360-4a82-9719-2cc21ac18089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:34.521598 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec85e2-4360-4a82-9719-2cc21ac18089 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:34.569903 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:34.580597 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d366f9-92ff-4bd1-b91f-2c0d32a15efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:34.581226 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d366f9-92ff-4bd1-b91f-2c0d32a15efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:34.581507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d366f9-92ff-4bd1-b91f-2c0d32a15efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:34.581787 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d366f9-92ff-4bd1-b91f-2c0d32a15efa admin admin] PATH_INFO: `/v3/projects/256c3d7b6e2a41d399d8105f76bacedc` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:34.583056 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d366f9-92ff-4bd1-b91f-2c0d32a15efa admin admin] RBAC: Authorizing `identity:delete_project(project_id=256c3d7b6e2a41d399d8105f76bacedc)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:34.585159 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d366f9-92ff-4bd1-b91f-2c0d32a15efa admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:34.630200 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-69d366f9-92ff-4bd1-b91f-2c0d32a15efa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '256c3d7b6e2a41d399d8105f76bacedc', 'request_id': 'req-69d366f9-92ff-4bd1-b91f-2c0d32a15efa'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:34.637006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-69d366f9-92ff-4bd1-b91f-2c0d32a15efa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '256c3d7b6e2a41d399d8105f76bacedc', 'request_id': 'req-69d366f9-92ff-4bd1-b91f-2c0d32a15efa'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:34.643122 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-69d366f9-92ff-4bd1-b91f-2c0d32a15efa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '256c3d7b6e2a41d399d8105f76bacedc', 'request_id': 'req-69d366f9-92ff-4bd1-b91f-2c0d32a15efa'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:34.654845 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-69d366f9-92ff-4bd1-b91f-2c0d32a15efa admin admin] The token cache is being invalidate because project 256c3d7b6e2a41d399d8105f76bacedc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 22:02:34.655132 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-69d366f9-92ff-4bd1-b91f-2c0d32a15efa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:34.655801 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-69d366f9-92ff-4bd1-b91f-2c0d32a15efa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:34.657284 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1293/2606] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 22:02:34 2022] DELETE /identity/v3/projects/256c3d7b6e2a41d399d8105f76bacedc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:34.663617 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a64342-ca85-4f6f-84e9-c45ae2fed5de None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:34.663906 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a64342-ca85-4f6f-84e9-c45ae2fed5de None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:34.664179 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a64342-ca85-4f6f-84e9-c45ae2fed5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:34.683554 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-c5a64342-ca85-4f6f-84e9-c45ae2fed5de None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:34.684801 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-c5a64342-ca85-4f6f-84e9-c45ae2fed5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:34.693928 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:34.745544 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1314/2607] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:34.753177 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f3ba6-3f0a-4ca6-bf63-c29d92fbb9bb None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:34.753397 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f3ba6-3f0a-4ca6-bf63-c29d92fbb9bb None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:34.753671 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f3ba6-3f0a-4ca6-bf63-c29d92fbb9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:34.773687 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-963f3ba6-3f0a-4ca6-bf63-c29d92fbb9bb None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:34.774876 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-963f3ba6-3f0a-4ca6-bf63-c29d92fbb9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:34.783257 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1294/2608] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 22:02:34.787260 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a64342-ca85-4f6f-84e9-c45ae2fed5de None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:34.796755 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a64342-ca85-4f6f-84e9-c45ae2fed5de None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:34.801371 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f94e8-e542-44f4-920d-76c2ab097742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:34.802001 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f94e8-e542-44f4-920d-76c2ab097742 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:34.802280 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f94e8-e542-44f4-920d-76c2ab097742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:34.802550 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f94e8-e542-44f4-920d-76c2ab097742 admin admin] PATH_INFO: `/v3/domains` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:34.803178 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f94e8-e542-44f4-920d-76c2ab097742 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 22:02:34.803478 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f94e8-e542-44f4-920d-76c2ab097742 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:34.805096 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f94e8-e542-44f4-920d-76c2ab097742 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:34.816191 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1315/2609] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 22:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 22:02:34.820328 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f3ba6-3f0a-4ca6-bf63-c29d92fbb9bb None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:34.829894 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f3ba6-3f0a-4ca6-bf63-c29d92fbb9bb None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:34.834459 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6b1a7-b0c2-4a71-9123-00132e8d50d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:34.835045 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6b1a7-b0c2-4a71-9123-00132e8d50d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:34.835323 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6b1a7-b0c2-4a71-9123-00132e8d50d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:34.835621 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6b1a7-b0c2-4a71-9123-00132e8d50d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:34.836260 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6b1a7-b0c2-4a71-9123-00132e8d50d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:34.838164 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6b1a7-b0c2-4a71-9123-00132e8d50d7 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:34.866649 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1295/2610] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 22:02:34 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:34.870975 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f94e8-e542-44f4-920d-76c2ab097742 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:34.880683 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f94e8-e542-44f4-920d-76c2ab097742 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:34.935949 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd050183-3ab7-4283-94e9-23e006867a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:34.936482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd050183-3ab7-4283-94e9-23e006867a89 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:34.936787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd050183-3ab7-4283-94e9-23e006867a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:34.937059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd050183-3ab7-4283-94e9-23e006867a89 admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:34.937708 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd050183-3ab7-4283-94e9-23e006867a89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:34.939678 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd050183-3ab7-4283-94e9-23e006867a89 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:34.961407 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1316/2611] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 22:02:34 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:34.965988 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6b1a7-b0c2-4a71-9123-00132e8d50d7 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:34.976087 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6b1a7-b0c2-4a71-9123-00132e8d50d7 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:34.980599 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29aac0-6422-4dc2-8f5b-3feb4603447f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:34.981179 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29aac0-6422-4dc2-8f5b-3feb4603447f admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:34.981452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29aac0-6422-4dc2-8f5b-3feb4603447f admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:34.981717 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29aac0-6422-4dc2-8f5b-3feb4603447f admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:34.982339 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29aac0-6422-4dc2-8f5b-3feb4603447f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:34.983973 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29aac0-6422-4dc2-8f5b-3feb4603447f admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:34.993599 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1296/2612] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:34.997870 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd050183-3ab7-4283-94e9-23e006867a89 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:35.007917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd050183-3ab7-4283-94e9-23e006867a89 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:35.013254 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6348063-0507-4e32-85a9-e1849b8734a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:35.013857 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6348063-0507-4e32-85a9-e1849b8734a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:35.014138 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6348063-0507-4e32-85a9-e1849b8734a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:35.014410 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6348063-0507-4e32-85a9-e1849b8734a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:35.015045 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6348063-0507-4e32-85a9-e1849b8734a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:35.016686 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6348063-0507-4e32-85a9-e1849b8734a2 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:35.027176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1317/2613] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:35.031105 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29aac0-6422-4dc2-8f5b-3feb4603447f admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:35.040957 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29aac0-6422-4dc2-8f5b-3feb4603447f admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:35.045490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6818c5-007f-4fb3-82da-6b1e6f536e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:35.046125 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6818c5-007f-4fb3-82da-6b1e6f536e87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:35.046396 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6818c5-007f-4fb3-82da-6b1e6f536e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:35.046666 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6818c5-007f-4fb3-82da-6b1e6f536e87 admin admin] PATH_INFO: `/v3/projects/f518c23dbb1f4a8e9b49065976543826/users/fdebed08b5d44dc48d9f57038ff7ca80/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:35.061932 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6818c5-007f-4fb3-82da-6b1e6f536e87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f518c23dbb1f4a8e9b49065976543826, user_id=fdebed08b5d44dc48d9f57038ff7ca80, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:35.064587 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6818c5-007f-4fb3-82da-6b1e6f536e87 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:35.077138 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1297/2614] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 22:02:35 2022] PUT /identity/v3/projects/f518c23dbb1f4a8e9b49065976543826/users/fdebed08b5d44dc48d9f57038ff7ca80/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:35.087304 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808a484-93fe-4f46-a325-02ab881e6a4d None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:35.087624 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808a484-93fe-4f46-a325-02ab881e6a4d None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:35.087904 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808a484-93fe-4f46-a325-02ab881e6a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:35.116064 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-9808a484-93fe-4f46-a325-02ab881e6a4d None None] MFA Rules not processed for user `fdebed08b5d44dc48d9f57038ff7ca80`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:35.117085 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-9808a484-93fe-4f46-a325-02ab881e6a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:35.180190 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1318/2615] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:35.185167 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6818c5-007f-4fb3-82da-6b1e6f536e87 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:35.243934 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6818c5-007f-4fb3-82da-6b1e6f536e87 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:35.248956 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b8274-dae5-4603-bb9b-77d95cf61d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:35.249564 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b8274-dae5-4603-bb9b-77d95cf61d59 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:35.249820 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b8274-dae5-4603-bb9b-77d95cf61d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:35.250086 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b8274-dae5-4603-bb9b-77d95cf61d59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:35.250696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b8274-dae5-4603-bb9b-77d95cf61d59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:35.252601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b8274-dae5-4603-bb9b-77d95cf61d59 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:35.279434 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1298/2616] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 22:02:35 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 22:02:35.283490 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808a484-93fe-4f46-a325-02ab881e6a4d None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:35.293939 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808a484-93fe-4f46-a325-02ab881e6a4d None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:35.301751 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:35.352846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd98ea3-44cf-4803-8daa-7248db50499d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:35.353454 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd98ea3-44cf-4803-8daa-7248db50499d admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:35.353734 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd98ea3-44cf-4803-8daa-7248db50499d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:35.354004 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd98ea3-44cf-4803-8daa-7248db50499d admin admin] PATH_INFO: `/v3/users` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:35.354631 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd98ea3-44cf-4803-8daa-7248db50499d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:35.357130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd98ea3-44cf-4803-8daa-7248db50499d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:35.377716 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1319/2617] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 22:02:35 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:35.382293 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b8274-dae5-4603-bb9b-77d95cf61d59 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:35.389049 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:35.393382 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b8274-dae5-4603-bb9b-77d95cf61d59 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:35.399563 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ad0e4-6f5c-4130-8c37-0faba1678ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:35.400379 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ad0e4-6f5c-4130-8c37-0faba1678ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:35.400827 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ad0e4-6f5c-4130-8c37-0faba1678ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:35.401274 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ad0e4-6f5c-4130-8c37-0faba1678ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:35.401913 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ad0e4-6f5c-4130-8c37-0faba1678ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:35.403585 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ad0e4-6f5c-4130-8c37-0faba1678ec7 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:35.414057 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1299/2618] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:35.418216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd98ea3-44cf-4803-8daa-7248db50499d admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:35.428519 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd98ea3-44cf-4803-8daa-7248db50499d admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:35.433087 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37934056-3603-44c7-bdc9-d45a9bc449bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:35.433726 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37934056-3603-44c7-bdc9-d45a9bc449bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:35.434002 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37934056-3603-44c7-bdc9-d45a9bc449bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:35.434795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37934056-3603-44c7-bdc9-d45a9bc449bc admin admin] PATH_INFO: `/v3/projects/7bed83e68a824f7dbbee1916388a06ef/users/5cab0e4a853843c887dab74ebea72257/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:35.449539 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37934056-3603-44c7-bdc9-d45a9bc449bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bed83e68a824f7dbbee1916388a06ef, user_id=5cab0e4a853843c887dab74ebea72257, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:35.452330 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37934056-3603-44c7-bdc9-d45a9bc449bc admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:35.464638 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1320/2619] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 22:02:35 2022] PUT /identity/v3/projects/7bed83e68a824f7dbbee1916388a06ef/users/5cab0e4a853843c887dab74ebea72257/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:35.468527 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ad0e4-6f5c-4130-8c37-0faba1678ec7 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:35.478371 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ad0e4-6f5c-4130-8c37-0faba1678ec7 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:35.533964 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eaefb3-c660-4208-8836-e1baf6cc458f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:35.534493 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eaefb3-c660-4208-8836-e1baf6cc458f admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:35.534772 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eaefb3-c660-4208-8836-e1baf6cc458f admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:35.535044 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eaefb3-c660-4208-8836-e1baf6cc458f admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:35.535703 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eaefb3-c660-4208-8836-e1baf6cc458f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:35.537278 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eaefb3-c660-4208-8836-e1baf6cc458f admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:35.547132 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1300/2620] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:35.551342 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37934056-3603-44c7-bdc9-d45a9bc449bc admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:35.562212 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37934056-3603-44c7-bdc9-d45a9bc449bc admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:35.566748 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60475d-2a0c-43cc-8109-23b003d38bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:35.567402 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60475d-2a0c-43cc-8109-23b003d38bbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:35.567718 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60475d-2a0c-43cc-8109-23b003d38bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:35.567994 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60475d-2a0c-43cc-8109-23b003d38bbf admin admin] PATH_INFO: `/v3/projects/7bed83e68a824f7dbbee1916388a06ef/users/5cab0e4a853843c887dab74ebea72257/roles/c514b065b7604094b899c586f79b0c5b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:35.569910 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60475d-2a0c-43cc-8109-23b003d38bbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bed83e68a824f7dbbee1916388a06ef, user_id=5cab0e4a853843c887dab74ebea72257, role_id=c514b065b7604094b899c586f79b0c5b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:35.572816 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60475d-2a0c-43cc-8109-23b003d38bbf admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:35.582853 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1321/2621] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 22:02:35 2022] PUT /identity/v3/projects/7bed83e68a824f7dbbee1916388a06ef/users/5cab0e4a853843c887dab74ebea72257/roles/c514b065b7604094b899c586f79b0c5b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:35.593657 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b521e2-afe6-4556-8763-3b1ef1e15c05 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:35.593943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b521e2-afe6-4556-8763-3b1ef1e15c05 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:35.594217 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b521e2-afe6-4556-8763-3b1ef1e15c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:35.621487 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-74b521e2-afe6-4556-8763-3b1ef1e15c05 None None] MFA Rules not processed for user `5cab0e4a853843c887dab74ebea72257`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:35.623288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-74b521e2-afe6-4556-8763-3b1ef1e15c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:35.688943 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1301/2622] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:35.700202 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60475d-2a0c-43cc-8109-23b003d38bbf admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:35.701963 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-7b60475d-2a0c-43cc-8109-23b003d38bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:35.759777 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60475d-2a0c-43cc-8109-23b003d38bbf admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:35.763428 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e73ea3-815d-4d22-b1dc-93ceb09a95c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:35.764059 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e73ea3-815d-4d22-b1dc-93ceb09a95c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:35.764340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e73ea3-815d-4d22-b1dc-93ceb09a95c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:35.764613 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e73ea3-815d-4d22-b1dc-93ceb09a95c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:35.818784 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e73ea3-815d-4d22-b1dc-93ceb09a95c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:35.820185 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e73ea3-815d-4d22-b1dc-93ceb09a95c9 service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:35.828906 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1322/2623] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:02:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:02:38.361477 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b521e2-afe6-4556-8763-3b1ef1e15c05 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:38.422431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b521e2-afe6-4556-8763-3b1ef1e15c05 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:38.426905 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5ee15-3c3b-4804-b2d2-34502215d4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:38.427522 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5ee15-3c3b-4804-b2d2-34502215d4b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:38.427847 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5ee15-3c3b-4804-b2d2-34502215d4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:38.428119 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5ee15-3c3b-4804-b2d2-34502215d4b9 admin admin] PATH_INFO: `/v3/users/fdebed08b5d44dc48d9f57038ff7ca80` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:38.429235 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5ee15-3c3b-4804-b2d2-34502215d4b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdebed08b5d44dc48d9f57038ff7ca80)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:38.431132 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5ee15-3c3b-4804-b2d2-34502215d4b9 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:38.447232 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:38.482706 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-16a5ee15-3c3b-4804-b2d2-34502215d4b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdebed08b5d44dc48d9f57038ff7ca80'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:38.483398 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-16a5ee15-3c3b-4804-b2d2-34502215d4b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdebed08b5d44dc48d9f57038ff7ca80'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:38.495531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-16a5ee15-3c3b-4804-b2d2-34502215d4b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdebed08b5d44dc48d9f57038ff7ca80'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:38.503531 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-16a5ee15-3c3b-4804-b2d2-34502215d4b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdebed08b5d44dc48d9f57038ff7ca80'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:38.504181 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-16a5ee15-3c3b-4804-b2d2-34502215d4b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdebed08b5d44dc48d9f57038ff7ca80'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:38.516669 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1302/2624] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 22:02:38 2022] DELETE /identity/v3/users/fdebed08b5d44dc48d9f57038ff7ca80 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:38.521193 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78286cb-d327-4fb0-a89c-adff79a47994 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:38.521675 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78286cb-d327-4fb0-a89c-adff79a47994 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:38.521773 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78286cb-d327-4fb0-a89c-adff79a47994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:38.540287 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-e78286cb-d327-4fb0-a89c-adff79a47994 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:38.541479 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e78286cb-d327-4fb0-a89c-adff79a47994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:38.553216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:38.600663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1323/2625] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:38.610732 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5ee15-3c3b-4804-b2d2-34502215d4b9 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:38.612180 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-16a5ee15-3c3b-4804-b2d2-34502215d4b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:38.668404 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5ee15-3c3b-4804-b2d2-34502215d4b9 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:38.672565 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c4d67-104c-4cf1-8eea-bc7e39360e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:38.673162 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c4d67-104c-4cf1-8eea-bc7e39360e33 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:38.673444 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c4d67-104c-4cf1-8eea-bc7e39360e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:38.673718 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c4d67-104c-4cf1-8eea-bc7e39360e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:38.682532 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c4d67-104c-4cf1-8eea-bc7e39360e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:38.683979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c4d67-104c-4cf1-8eea-bc7e39360e33 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:38.693273 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1303/2626] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:02:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:02:38.953572 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78286cb-d327-4fb0-a89c-adff79a47994 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:38.955188 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e78286cb-d327-4fb0-a89c-adff79a47994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:38.967974 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78286cb-d327-4fb0-a89c-adff79a47994 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:38.973104 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:38.973787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:38.974076 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:38.974830 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec admin admin] PATH_INFO: `/v3/projects/f518c23dbb1f4a8e9b49065976543826` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:38.976180 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=f518c23dbb1f4a8e9b49065976543826)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:38.978068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:39.023982 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f518c23dbb1f4a8e9b49065976543826', 'request_id': 'req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:39.029957 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f518c23dbb1f4a8e9b49065976543826', 'request_id': 'req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:39.037983 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f518c23dbb1f4a8e9b49065976543826', 'request_id': 'req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:39.048395 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec admin admin] The token cache is being invalidate because project f518c23dbb1f4a8e9b49065976543826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 22:02:39.048680 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:39.049297 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:39.050763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1324/2627] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 22:02:38 2022] DELETE /identity/v3/projects/f518c23dbb1f4a8e9b49065976543826 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:39.054899 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c4d67-104c-4cf1-8eea-bc7e39360e33 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:39.064482 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c4d67-104c-4cf1-8eea-bc7e39360e33 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:39.118606 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e728348-6556-4a81-91fa-6e6532674ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:39.119223 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e728348-6556-4a81-91fa-6e6532674ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:39.119500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e728348-6556-4a81-91fa-6e6532674ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:39.119800 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e728348-6556-4a81-91fa-6e6532674ad5 admin admin] PATH_INFO: `/v3/users/5cab0e4a853843c887dab74ebea72257` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:39.120909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e728348-6556-4a81-91fa-6e6532674ad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cab0e4a853843c887dab74ebea72257)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:39.122803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e728348-6556-4a81-91fa-6e6532674ad5 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:39.162783 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:39.169804 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-2e728348-6556-4a81-91fa-6e6532674ad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cab0e4a853843c887dab74ebea72257'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:39.170464 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-2e728348-6556-4a81-91fa-6e6532674ad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cab0e4a853843c887dab74ebea72257'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:39.182541 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-2e728348-6556-4a81-91fa-6e6532674ad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cab0e4a853843c887dab74ebea72257'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:39.189364 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-2e728348-6556-4a81-91fa-6e6532674ad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cab0e4a853843c887dab74ebea72257'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:39.190006 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-2e728348-6556-4a81-91fa-6e6532674ad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cab0e4a853843c887dab74ebea72257'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:39.201707 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1304/2628] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 22:02:39 2022] DELETE /identity/v3/users/5cab0e4a853843c887dab74ebea72257 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:39.463938 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:39.465431 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:39.479562 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5398e6-fc19-464a-ba2f-b8fbda0326ec admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:39.535488 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c234c-afda-465d-89d1-701674f46c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:39.536160 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c234c-afda-465d-89d1-701674f46c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:39.536414 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c234c-afda-465d-89d1-701674f46c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:39.536681 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c234c-afda-465d-89d1-701674f46c4b admin admin] PATH_INFO: `/v3/projects/7bed83e68a824f7dbbee1916388a06ef` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:39.537779 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c234c-afda-465d-89d1-701674f46c4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bed83e68a824f7dbbee1916388a06ef)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:39.539713 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c234c-afda-465d-89d1-701674f46c4b admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:39.581761 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:39.585846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-075c234c-afda-465d-89d1-701674f46c4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bed83e68a824f7dbbee1916388a06ef', 'request_id': 'req-075c234c-afda-465d-89d1-701674f46c4b'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:39.593354 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-075c234c-afda-465d-89d1-701674f46c4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bed83e68a824f7dbbee1916388a06ef', 'request_id': 'req-075c234c-afda-465d-89d1-701674f46c4b'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:39.600437 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-075c234c-afda-465d-89d1-701674f46c4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bed83e68a824f7dbbee1916388a06ef', 'request_id': 'req-075c234c-afda-465d-89d1-701674f46c4b'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:39.610529 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-075c234c-afda-465d-89d1-701674f46c4b admin admin] The token cache is being invalidate because project 7bed83e68a824f7dbbee1916388a06ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 22:02:39.610815 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-075c234c-afda-465d-89d1-701674f46c4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:39.611439 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-075c234c-afda-465d-89d1-701674f46c4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:02:39.612934 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1325/2629] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 22:02:39 2022] DELETE /identity/v3/projects/7bed83e68a824f7dbbee1916388a06ef => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:39.617733 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db7448-e198-4622-bf2a-7d5820f0b3f6 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:39.618209 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db7448-e198-4622-bf2a-7d5820f0b3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:39.618416 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db7448-e198-4622-bf2a-7d5820f0b3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:39.639609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-41db7448-e198-4622-bf2a-7d5820f0b3f6 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:39.641480 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-41db7448-e198-4622-bf2a-7d5820f0b3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:39.698312 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1305/2630] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:02:39.707003 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d82a4e-b091-4701-bcee-fba272aa2a14 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:39.707294 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d82a4e-b091-4701-bcee-fba272aa2a14 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:39.707595 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d82a4e-b091-4701-bcee-fba272aa2a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:39.728176 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-92d82a4e-b091-4701-bcee-fba272aa2a14 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:39.729382 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-92d82a4e-b091-4701-bcee-fba272aa2a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:39.737613 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1326/2631] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 22:02:39.741551 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db7448-e198-4622-bf2a-7d5820f0b3f6 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:39.751747 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db7448-e198-4622-bf2a-7d5820f0b3f6 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:39.756641 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd538c-e63a-4570-9a86-fa47dd8f4e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:39.757231 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd538c-e63a-4570-9a86-fa47dd8f4e94 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:39.757507 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd538c-e63a-4570-9a86-fa47dd8f4e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:39.757776 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd538c-e63a-4570-9a86-fa47dd8f4e94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:39.758411 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd538c-e63a-4570-9a86-fa47dd8f4e94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 22:02:39.758712 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd538c-e63a-4570-9a86-fa47dd8f4e94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:39.760350 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd538c-e63a-4570-9a86-fa47dd8f4e94 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:39.772189 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1306/2632] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 22:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 22:02:39.776453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d82a4e-b091-4701-bcee-fba272aa2a14 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:39.785589 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d82a4e-b091-4701-bcee-fba272aa2a14 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:39.790195 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0219800-f7ae-489b-8f47-128cdd302511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:39.790787 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0219800-f7ae-489b-8f47-128cdd302511 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:39.791068 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0219800-f7ae-489b-8f47-128cdd302511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:39.791341 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0219800-f7ae-489b-8f47-128cdd302511 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:39.792011 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0219800-f7ae-489b-8f47-128cdd302511 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:39.793924 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0219800-f7ae-489b-8f47-128cdd302511 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:39.821259 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1327/2633] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 22:02:39 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:02:39.825431 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd538c-e63a-4570-9a86-fa47dd8f4e94 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:39.835753 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd538c-e63a-4570-9a86-fa47dd8f4e94 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:39.892021 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7ac4b-dd8d-4249-9b23-3b7a603e2721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:39.892609 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7ac4b-dd8d-4249-9b23-3b7a603e2721 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:39.892884 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7ac4b-dd8d-4249-9b23-3b7a603e2721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:39.893155 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7ac4b-dd8d-4249-9b23-3b7a603e2721 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:39.893768 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7ac4b-dd8d-4249-9b23-3b7a603e2721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:39.895667 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7ac4b-dd8d-4249-9b23-3b7a603e2721 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:39.916296 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1307/2634] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 22:02:39 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 22:02:39.920372 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0219800-f7ae-489b-8f47-128cdd302511 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:39.931130 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0219800-f7ae-489b-8f47-128cdd302511 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:39.935858 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a18b2-d39c-45ba-8787-52dc3370d024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:39.936484 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a18b2-d39c-45ba-8787-52dc3370d024 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:39.936763 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a18b2-d39c-45ba-8787-52dc3370d024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:39.937035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a18b2-d39c-45ba-8787-52dc3370d024 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:39.938225 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a18b2-d39c-45ba-8787-52dc3370d024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:39.939846 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a18b2-d39c-45ba-8787-52dc3370d024 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:39.950151 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1328/2635] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:39.953946 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7ac4b-dd8d-4249-9b23-3b7a603e2721 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:39.963927 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7ac4b-dd8d-4249-9b23-3b7a603e2721 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:39.968496 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca17df8-8c37-49ab-b9a2-02c3eebef810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:39.969101 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca17df8-8c37-49ab-b9a2-02c3eebef810 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:39.969373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca17df8-8c37-49ab-b9a2-02c3eebef810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:39.969640 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca17df8-8c37-49ab-b9a2-02c3eebef810 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:39.970257 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca17df8-8c37-49ab-b9a2-02c3eebef810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:39.971831 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca17df8-8c37-49ab-b9a2-02c3eebef810 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:39.982038 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1308/2636] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:02:39.986604 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a18b2-d39c-45ba-8787-52dc3370d024 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:39.996273 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a18b2-d39c-45ba-8787-52dc3370d024 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:40.001615 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7aa84-c591-4383-9ccf-c0e784ded1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:40.002262 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7aa84-c591-4383-9ccf-c0e784ded1c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:40.002542 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7aa84-c591-4383-9ccf-c0e784ded1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:40.002815 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7aa84-c591-4383-9ccf-c0e784ded1c2 admin admin] PATH_INFO: `/v3/projects/f79cb5859d444082808f3e6935efc9e1/users/90b1f5fbf51b4bbbb1556b437c550871/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:40.017332 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7aa84-c591-4383-9ccf-c0e784ded1c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f79cb5859d444082808f3e6935efc9e1, user_id=90b1f5fbf51b4bbbb1556b437c550871, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:40.020063 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7aa84-c591-4383-9ccf-c0e784ded1c2 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:40.032716 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1329/2637] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 22:02:39 2022] PUT /identity/v3/projects/f79cb5859d444082808f3e6935efc9e1/users/90b1f5fbf51b4bbbb1556b437c550871/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:02:40.042861 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba7c45-e566-47fb-a209-981b57c4d3ef None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:40.043147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba7c45-e566-47fb-a209-981b57c4d3ef None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:40.043417 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba7c45-e566-47fb-a209-981b57c4d3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:40.071293 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-dcba7c45-e566-47fb-a209-981b57c4d3ef None None] MFA Rules not processed for user `90b1f5fbf51b4bbbb1556b437c550871`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:02:40.072518 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-dcba7c45-e566-47fb-a209-981b57c4d3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:40.081500 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:40.139362 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1309/2638] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:02:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 22:02:40.151228 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7aa84-c591-4383-9ccf-c0e784ded1c2 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:02:40.152918 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-abc7aa84-c591-4383-9ccf-c0e784ded1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:02:40.207789 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7aa84-c591-4383-9ccf-c0e784ded1c2 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:02:40.211314 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4457d5b-509c-41df-a058-b3bff15e11a4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ce5ec441b06408bbadd6fc040af0da5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:02:40.211938 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4457d5b-509c-41df-a058-b3bff15e11a4 service glance] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:02:40.212216 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4457d5b-509c-41df-a058-b3bff15e11a4 service glance] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:02:40.212489 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4457d5b-509c-41df-a058-b3bff15e11a4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:02:40.221018 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4457d5b-509c-41df-a058-b3bff15e11a4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:02:40.222453 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4457d5b-509c-41df-a058-b3bff15e11a4 service glance] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:02:40.230931 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1330/2639] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:02:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 22:02:50.136444 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=22 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:02:50.227638 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=61,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:03:28.272997 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba7c45-e566-47fb-a209-981b57c4d3ef None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:28.352003 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba7c45-e566-47fb-a209-981b57c4d3ef None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:28.357213 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b5703-f55a-4093-b0f3-3e1d84531911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:28.357835 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b5703-f55a-4093-b0f3-3e1d84531911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:28.358111 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b5703-f55a-4093-b0f3-3e1d84531911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:28.358378 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b5703-f55a-4093-b0f3-3e1d84531911 admin admin] PATH_INFO: `/v3/users/90b1f5fbf51b4bbbb1556b437c550871` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:28.359469 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b5703-f55a-4093-b0f3-3e1d84531911 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90b1f5fbf51b4bbbb1556b437c550871)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:28.361419 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b5703-f55a-4093-b0f3-3e1d84531911 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:28.406570 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-464b5703-f55a-4093-b0f3-3e1d84531911 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90b1f5fbf51b4bbbb1556b437c550871'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:28.407236 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-464b5703-f55a-4093-b0f3-3e1d84531911 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90b1f5fbf51b4bbbb1556b437c550871'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:28.419352 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-464b5703-f55a-4093-b0f3-3e1d84531911 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90b1f5fbf51b4bbbb1556b437c550871'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:28.426306 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-464b5703-f55a-4093-b0f3-3e1d84531911 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90b1f5fbf51b4bbbb1556b437c550871'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:28.426933 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-464b5703-f55a-4093-b0f3-3e1d84531911 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90b1f5fbf51b4bbbb1556b437c550871'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:28.438631 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1310/2640] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 22:03:28 2022] DELETE /identity/v3/users/90b1f5fbf51b4bbbb1556b437c550871 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 22:03:28.443183 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b69ae-a950-4789-9a60-15fabb8ce1c4 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:28.443688 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b69ae-a950-4789-9a60-15fabb8ce1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:28.443795 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b69ae-a950-4789-9a60-15fabb8ce1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:28.474268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-2f6b69ae-a950-4789-9a60-15fabb8ce1c4 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:03:28.475492 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2f6b69ae-a950-4789-9a60-15fabb8ce1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:28.535021 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1331/2641] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:03:28.545997 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b5703-f55a-4093-b0f3-3e1d84531911 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:28.548118 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-464b5703-f55a-4093-b0f3-3e1d84531911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:28.609451 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b5703-f55a-4093-b0f3-3e1d84531911 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:28.613095 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24575367-ae49-44e2-9a9f-82f6b7cf4843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:28.613703 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24575367-ae49-44e2-9a9f-82f6b7cf4843 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:28.613978 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24575367-ae49-44e2-9a9f-82f6b7cf4843 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:28.614247 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24575367-ae49-44e2-9a9f-82f6b7cf4843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:28.623310 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24575367-ae49-44e2-9a9f-82f6b7cf4843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:28.624767 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24575367-ae49-44e2-9a9f-82f6b7cf4843 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:28.634752 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1311/2642] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:03:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:03:29.060885 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b69ae-a950-4789-9a60-15fabb8ce1c4 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:29.062485 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-2f6b69ae-a950-4789-9a60-15fabb8ce1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:29.077729 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b69ae-a950-4789-9a60-15fabb8ce1c4 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:29.082650 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bfc9da-ac7d-4d66-8cb5-51dadeeee3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:29.083288 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfc9da-ac7d-4d66-8cb5-51dadeeee3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:29.083589 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfc9da-ac7d-4d66-8cb5-51dadeeee3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:29.083868 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bfc9da-ac7d-4d66-8cb5-51dadeeee3e6 admin admin] PATH_INFO: `/v3/projects/f79cb5859d444082808f3e6935efc9e1` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:29.084995 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfc9da-ac7d-4d66-8cb5-51dadeeee3e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f79cb5859d444082808f3e6935efc9e1)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:29.086983 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bfc9da-ac7d-4d66-8cb5-51dadeeee3e6 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:29.142533 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-52bfc9da-ac7d-4d66-8cb5-51dadeeee3e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f79cb5859d444082808f3e6935efc9e1', 'request_id': 'req-52bfc9da-ac7d-4d66-8cb5-51dadeeee3e6'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:29.148660 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-52bfc9da-ac7d-4d66-8cb5-51dadeeee3e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f79cb5859d444082808f3e6935efc9e1', 'request_id': 'req-52bfc9da-ac7d-4d66-8cb5-51dadeeee3e6'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:29.156793 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-52bfc9da-ac7d-4d66-8cb5-51dadeeee3e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f79cb5859d444082808f3e6935efc9e1', 'request_id': 'req-52bfc9da-ac7d-4d66-8cb5-51dadeeee3e6'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:29.166943 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-52bfc9da-ac7d-4d66-8cb5-51dadeeee3e6 admin admin] The token cache is being invalidate because project f79cb5859d444082808f3e6935efc9e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 22:03:29.167227 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-52bfc9da-ac7d-4d66-8cb5-51dadeeee3e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:29.167861 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-52bfc9da-ac7d-4d66-8cb5-51dadeeee3e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:29.169373 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1332/2643] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 22:03:29 2022] DELETE /identity/v3/projects/f79cb5859d444082808f3e6935efc9e1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:03:29.174136 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c0b8-346e-4eea-bc5a-c2ba6c64437a None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:29.174416 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c0b8-346e-4eea-bc5a-c2ba6c64437a None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:29.174686 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c0b8-346e-4eea-bc5a-c2ba6c64437a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:29.183685 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:03:29.196514 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-9b15c0b8-346e-4eea-bc5a-c2ba6c64437a None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:03:29.198789 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-9b15c0b8-346e-4eea-bc5a-c2ba6c64437a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:29.261025 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1312/2644] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 22:03:29.268974 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11034df1-6364-4403-b76b-34f6d02893f3 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:29.269257 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11034df1-6364-4403-b76b-34f6d02893f3 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:29.269549 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11034df1-6364-4403-b76b-34f6d02893f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:29.289268 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-11034df1-6364-4403-b76b-34f6d02893f3 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:03:29.290499 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-11034df1-6364-4403-b76b-34f6d02893f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:29.298458 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1333/2645] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:03:29.302747 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15c0b8-346e-4eea-bc5a-c2ba6c64437a None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:29.313012 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15c0b8-346e-4eea-bc5a-c2ba6c64437a None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:29.317449 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48159d9-02fd-4f6e-9247-0651c7633902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:29.318041 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48159d9-02fd-4f6e-9247-0651c7633902 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:29.318319 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48159d9-02fd-4f6e-9247-0651c7633902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:29.318617 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48159d9-02fd-4f6e-9247-0651c7633902 admin admin] PATH_INFO: `/v3/domains` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:29.319260 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48159d9-02fd-4f6e-9247-0651c7633902 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 22:03:29.319590 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48159d9-02fd-4f6e-9247-0651c7633902 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:29.321161 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48159d9-02fd-4f6e-9247-0651c7633902 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:29.333028 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1313/2646] 10.222.0.13 () {58 vars in 1153 bytes} [Fri Oct 21 22:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 22:03:29.337225 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11034df1-6364-4403-b76b-34f6d02893f3 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:29.346266 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11034df1-6364-4403-b76b-34f6d02893f3 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:29.351053 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9afe77-4119-4fd8-8912-f15f4dec0530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:29.351668 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9afe77-4119-4fd8-8912-f15f4dec0530 admin admin] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:29.351946 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9afe77-4119-4fd8-8912-f15f4dec0530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:29.352215 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9afe77-4119-4fd8-8912-f15f4dec0530 admin admin] PATH_INFO: `/v3/projects` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:29.352866 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9afe77-4119-4fd8-8912-f15f4dec0530 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:29.355281 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9afe77-4119-4fd8-8912-f15f4dec0530 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:29.382440 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1334/2647] 10.222.0.13 () {60 vars in 1153 bytes} [Fri Oct 21 22:03:29 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:03:29.386608 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48159d9-02fd-4f6e-9247-0651c7633902 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:29.396399 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48159d9-02fd-4f6e-9247-0651c7633902 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:29.451991 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40bc36-9f61-45bf-85c0-02835c3b7f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:29.452590 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40bc36-9f61-45bf-85c0-02835c3b7f51 admin admin] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:29.452870 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40bc36-9f61-45bf-85c0-02835c3b7f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:29.453143 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40bc36-9f61-45bf-85c0-02835c3b7f51 admin admin] PATH_INFO: `/v3/users` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:29.453773 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f40bc36-9f61-45bf-85c0-02835c3b7f51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:29.455759 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f40bc36-9f61-45bf-85c0-02835c3b7f51 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:29.476343 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1314/2648] 10.222.0.13 () {60 vars in 1144 bytes} [Fri Oct 21 22:03:29 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 22:03:29.481035 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9afe77-4119-4fd8-8912-f15f4dec0530 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:29.491150 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9afe77-4119-4fd8-8912-f15f4dec0530 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:29.495757 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb6fe6-063c-4702-96ec-9621b3866978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:29.496346 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb6fe6-063c-4702-96ec-9621b3866978 admin admin] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:29.496626 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb6fe6-063c-4702-96ec-9621b3866978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:29.496892 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb6fe6-063c-4702-96ec-9621b3866978 admin admin] PATH_INFO: `/v3/roles` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:29.497519 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb6fe6-063c-4702-96ec-9621b3866978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:29.499258 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb6fe6-063c-4702-96ec-9621b3866978 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:29.509262 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1335/2649] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 22:03:29.513745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40bc36-9f61-45bf-85c0-02835c3b7f51 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:29.523643 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40bc36-9f61-45bf-85c0-02835c3b7f51 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:29.528696 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67019f5-c3fa-42c7-a863-7da8f61b86d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:29.529284 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67019f5-c3fa-42c7-a863-7da8f61b86d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:29.529564 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67019f5-c3fa-42c7-a863-7da8f61b86d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:29.529835 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67019f5-c3fa-42c7-a863-7da8f61b86d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:29.530458 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67019f5-c3fa-42c7-a863-7da8f61b86d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:29.532056 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67019f5-c3fa-42c7-a863-7da8f61b86d5 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:29.543304 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1315/2650] 10.222.0.13 () {58 vars in 1122 bytes} [Fri Oct 21 22:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 22:03:29.547168 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb6fe6-063c-4702-96ec-9621b3866978 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:29.556721 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb6fe6-063c-4702-96ec-9621b3866978 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:29.561187 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135b942-2332-43ac-ad62-949f36e6e20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:29.561815 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135b942-2332-43ac-ad62-949f36e6e20d admin admin] REQUEST_METHOD: `PUT` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:29.562092 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135b942-2332-43ac-ad62-949f36e6e20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:29.562360 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135b942-2332-43ac-ad62-949f36e6e20d admin admin] PATH_INFO: `/v3/projects/96921b84c76a4584b5a840d224f3688b/users/2d4d6814723f473cb2ae176a77e286d2/roles/c73d9ac0c74147e095a9a038d200e2e9` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:29.576950 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135b942-2332-43ac-ad62-949f36e6e20d admin admin] RBAC: Authorizing `identity:create_grant(project_id=96921b84c76a4584b5a840d224f3688b, user_id=2d4d6814723f473cb2ae176a77e286d2, role_id=c73d9ac0c74147e095a9a038d200e2e9)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:29.579510 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135b942-2332-43ac-ad62-949f36e6e20d admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:29.592487 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1336/2651] 10.222.0.13 () {60 vars in 1483 bytes} [Fri Oct 21 22:03:29 2022] PUT /identity/v3/projects/96921b84c76a4584b5a840d224f3688b/users/2d4d6814723f473cb2ae176a77e286d2/roles/c73d9ac0c74147e095a9a038d200e2e9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:03:29.602803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21f2a8-3c8f-45d5-a171-d8d614462490 None None] REQUEST_METHOD: `POST` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:29.603092 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21f2a8-3c8f-45d5-a171-d8d614462490 None None] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:29.603373 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21f2a8-3c8f-45d5-a171-d8d614462490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:29.630686 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.auth.core [None req-1d21f2a8-3c8f-45d5-a171-d8d614462490 None None] MFA Rules not processed for user `2d4d6814723f473cb2ae176a77e286d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:03:29.631911 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-1d21f2a8-3c8f-45d5-a171-d8d614462490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:29.690204 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:03:29.698715 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1316/2652] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:03:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:03:29.705992 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135b942-2332-43ac-ad62-949f36e6e20d admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:29.707663 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-1135b942-2332-43ac-ad62-949f36e6e20d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:29.718836 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:03:29.775806 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135b942-2332-43ac-ad62-949f36e6e20d admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:29.779375 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff041f9-612b-459a-80c8-c17c5b2a2b5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd620cad0ef21453a9cab61f728913cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:29.779998 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff041f9-612b-459a-80c8-c17c5b2a2b5a service cinder] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:29.780275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff041f9-612b-459a-80c8-c17c5b2a2b5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:29.780541 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff041f9-612b-459a-80c8-c17c5b2a2b5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:29.788534 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff041f9-612b-459a-80c8-c17c5b2a2b5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:29.789948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff041f9-612b-459a-80c8-c17c5b2a2b5a service cinder] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:29.798532 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1337/2653] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:03:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 22:03:32.339030 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21f2a8-3c8f-45d5-a171-d8d614462490 None None] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:32.399601 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21f2a8-3c8f-45d5-a171-d8d614462490 None None] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:32.404025 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a270ca-6452-4a99-86d7-2302391f0f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:32.404635 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a270ca-6452-4a99-86d7-2302391f0f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:32.404909 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a270ca-6452-4a99-86d7-2302391f0f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:32.405177 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a270ca-6452-4a99-86d7-2302391f0f69 admin admin] PATH_INFO: `/v3/users/2d4d6814723f473cb2ae176a77e286d2` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:32.406282 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a270ca-6452-4a99-86d7-2302391f0f69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d4d6814723f473cb2ae176a77e286d2)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:32.408197 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a270ca-6452-4a99-86d7-2302391f0f69 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:32.452288 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a9a270ca-6452-4a99-86d7-2302391f0f69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d4d6814723f473cb2ae176a77e286d2'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:32.452918 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a9a270ca-6452-4a99-86d7-2302391f0f69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d4d6814723f473cb2ae176a77e286d2'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:32.464446 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a9a270ca-6452-4a99-86d7-2302391f0f69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d4d6814723f473cb2ae176a77e286d2'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:32.471465 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a9a270ca-6452-4a99-86d7-2302391f0f69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d4d6814723f473cb2ae176a77e286d2'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:32.472152 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-a9a270ca-6452-4a99-86d7-2302391f0f69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d4d6814723f473cb2ae176a77e286d2'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:32.484775 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1317/2654] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 22:03:32 2022] DELETE /identity/v3/users/2d4d6814723f473cb2ae176a77e286d2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:03:32.489206 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb7abe-6955-4efc-af82-fe6a22a66c76 None None] REQUEST_METHOD: `POST` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:32.489670 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb7abe-6955-4efc-af82-fe6a22a66c76 None None] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:32.489768 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb7abe-6955-4efc-af82-fe6a22a66c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:32.509161 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.auth.core [None req-f5eb7abe-6955-4efc-af82-fe6a22a66c76 None None] MFA Rules not processed for user `85a2a3bc91ec4aebbb1e136167a1d87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 22:03:32.510357 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f5eb7abe-6955-4efc-af82-fe6a22a66c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:32.568374 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1338/2655] 10.222.0.13 () {58 vars in 958 bytes} [Fri Oct 21 22:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 22:03:32.577979 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a270ca-6452-4a99-86d7-2302391f0f69 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:32.579408 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a9a270ca-6452-4a99-86d7-2302391f0f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:32.635856 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a270ca-6452-4a99-86d7-2302391f0f69 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:32.639462 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74385987-3ab9-4070-92e5-766fba855be9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:32.640089 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74385987-3ab9-4070-92e5-766fba855be9 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:32.640372 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74385987-3ab9-4070-92e5-766fba855be9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:32.640653 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74385987-3ab9-4070-92e5-766fba855be9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:32.649791 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74385987-3ab9-4070-92e5-766fba855be9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:32.650982 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74385987-3ab9-4070-92e5-766fba855be9 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:32.660764 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1318/2656] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:03:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:03:32.926737 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb7abe-6955-4efc-af82-fe6a22a66c76 None None] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:32.928183 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-f5eb7abe-6955-4efc-af82-fe6a22a66c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:32.941931 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb7abe-6955-4efc-af82-fe6a22a66c76 None None] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:32.946117 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cea526-5187-45db-ad85-c69509d27285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:32.946746 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cea526-5187-45db-ad85-c69509d27285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:32.947600 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cea526-5187-45db-ad85-c69509d27285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:32.947880 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cea526-5187-45db-ad85-c69509d27285 admin admin] PATH_INFO: `/v3/projects/96921b84c76a4584b5a840d224f3688b` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:32.949191 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cea526-5187-45db-ad85-c69509d27285 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96921b84c76a4584b5a840d224f3688b)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:32.951116 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cea526-5187-45db-ad85-c69509d27285 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:32.996623 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-80cea526-5187-45db-ad85-c69509d27285 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96921b84c76a4584b5a840d224f3688b', 'request_id': 'req-80cea526-5187-45db-ad85-c69509d27285'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:33.002780 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-80cea526-5187-45db-ad85-c69509d27285 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96921b84c76a4584b5a840d224f3688b', 'request_id': 'req-80cea526-5187-45db-ad85-c69509d27285'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:33.010271 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-80cea526-5187-45db-ad85-c69509d27285 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96921b84c76a4584b5a840d224f3688b', 'request_id': 'req-80cea526-5187-45db-ad85-c69509d27285'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:33.020517 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-80cea526-5187-45db-ad85-c69509d27285 admin admin] The token cache is being invalidate because project 96921b84c76a4584b5a840d224f3688b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 22:03:33.020806 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-80cea526-5187-45db-ad85-c69509d27285 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:33.021432 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-80cea526-5187-45db-ad85-c69509d27285 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:03:33.022906 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1339/2657] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 22:03:32 2022] DELETE /identity/v3/projects/96921b84c76a4584b5a840d224f3688b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:03:42.656505 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:03:43.017543 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:03:53.778904 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74385987-3ab9-4070-92e5-766fba855be9 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:53.780348 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-74385987-3ab9-4070-92e5-766fba855be9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:53.791616 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74385987-3ab9-4070-92e5-766fba855be9 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:53.839328 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20015418-4ccc-4378-81e1-307f352e3be6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:53.839952 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20015418-4ccc-4378-81e1-307f352e3be6 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:53.840235 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20015418-4ccc-4378-81e1-307f352e3be6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:53.840508 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20015418-4ccc-4378-81e1-307f352e3be6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:53.842190 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-20015418-4ccc-4378-81e1-307f352e3be6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:53.899270 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20015418-4ccc-4378-81e1-307f352e3be6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:53.900745 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20015418-4ccc-4378-81e1-307f352e3be6 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:53.909607 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1319/2658] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:03:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:03:55.005275 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cea526-5187-45db-ad85-c69509d27285 admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:55.007749 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-80cea526-5187-45db-ad85-c69509d27285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:55.107344 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cea526-5187-45db-ad85-c69509d27285 admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:55.113162 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696468fa-7cb4-48fd-9eff-43570fbf51be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12dad9f016a740b98538722e1b723823', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:55.113808 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696468fa-7cb4-48fd-9eff-43570fbf51be service nova] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:55.114096 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696468fa-7cb4-48fd-9eff-43570fbf51be service nova] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:55.114373 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696468fa-7cb4-48fd-9eff-43570fbf51be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:55.116442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-696468fa-7cb4-48fd-9eff-43570fbf51be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:55.130581 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696468fa-7cb4-48fd-9eff-43570fbf51be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:55.133084 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696468fa-7cb4-48fd-9eff-43570fbf51be service nova] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:55.147641 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1340/2659] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:03:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:03:55.221452 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20015418-4ccc-4378-81e1-307f352e3be6 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:55.222888 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-20015418-4ccc-4378-81e1-307f352e3be6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:55.233836 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20015418-4ccc-4378-81e1-307f352e3be6 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:55.237497 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f427a82-f9d8-42f2-865e-d1e57c5064e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:55.238155 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f427a82-f9d8-42f2-865e-d1e57c5064e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:55.238377 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f427a82-f9d8-42f2-865e-d1e57c5064e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:55.238653 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f427a82-f9d8-42f2-865e-d1e57c5064e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:55.240788 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-4f427a82-f9d8-42f2-865e-d1e57c5064e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:55.255215 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f427a82-f9d8-42f2-865e-d1e57c5064e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:55.256669 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f427a82-f9d8-42f2-865e-d1e57c5064e8 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:55.268393 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1320/2660] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:03:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:03:56.447455 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696468fa-7cb4-48fd-9eff-43570fbf51be service nova] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:56.448891 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-696468fa-7cb4-48fd-9eff-43570fbf51be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:56.460091 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696468fa-7cb4-48fd-9eff-43570fbf51be service nova] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:56.464110 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442a725-72a3-442a-9937-6b70a1b30ac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:56.464697 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442a725-72a3-442a-9937-6b70a1b30ac3 service neutron] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:56.464971 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442a725-72a3-442a-9937-6b70a1b30ac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:56.465234 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442a725-72a3-442a-9937-6b70a1b30ac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:56.466897 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e442a725-72a3-442a-9937-6b70a1b30ac3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:56.480380 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442a725-72a3-442a-9937-6b70a1b30ac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:56.481814 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442a725-72a3-442a-9937-6b70a1b30ac3 service neutron] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:56.493083 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1341/2661] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:03:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:03:58.998760 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f427a82-f9d8-42f2-865e-d1e57c5064e8 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:03:59.007923 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f427a82-f9d8-42f2-865e-d1e57c5064e8 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:03:59.011632 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfb7fc-3922-481c-843a-4880d5d0a4b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34dc4e7c5474b6b96977019328d8890', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:03:59.012234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfb7fc-3922-481c-843a-4880d5d0a4b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:03:59.012517 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfb7fc-3922-481c-843a-4880d5d0a4b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:03:59.012798 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfb7fc-3922-481c-843a-4880d5d0a4b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:03:59.014490 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-85bfb7fc-3922-481c-843a-4880d5d0a4b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:03:59.076532 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfb7fc-3922-481c-843a-4880d5d0a4b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:03:59.077947 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfb7fc-3922-481c-843a-4880d5d0a4b1 service neutron] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:03:59.087367 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1321/2662] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:03:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:04:05.280452 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442a725-72a3-442a-9937-6b70a1b30ac3 service neutron] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:04:05.281878 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-e442a725-72a3-442a-9937-6b70a1b30ac3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:04:05.356249 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfb7fc-3922-481c-843a-4880d5d0a4b1 service neutron] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:04:05.358299 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-85bfb7fc-3922-481c-843a-4880d5d0a4b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:04:05.365218 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442a725-72a3-442a-9937-6b70a1b30ac3 service neutron] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:04:05.371867 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ad3dd-c3ff-4547-acd3-74e4d852df91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:04:05.372484 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ad3dd-c3ff-4547-acd3-74e4d852df91 service placement] REQUEST_METHOD: `GET` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:04:05.372765 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ad3dd-c3ff-4547-acd3-74e4d852df91 service placement] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:04:05.373037 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ad3dd-c3ff-4547-acd3-74e4d852df91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:04:05.373193 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfb7fc-3922-481c-843a-4880d5d0a4b1 service neutron] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:04:05.374764 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-118ad3dd-c3ff-4547-acd3-74e4d852df91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:04:05.377679 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353f650-f80f-43f6-b648-ee3f2d0c2dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:04:05.378292 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353f650-f80f-43f6-b648-ee3f2d0c2dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:04:05.378565 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353f650-f80f-43f6-b648-ee3f2d0c2dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:04:05.378835 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353f650-f80f-43f6-b648-ee3f2d0c2dd5 admin admin] PATH_INFO: `/v3/users/05ec7e8b5ae04b2d84323ececfbf5acc` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:04:05.380243 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6353f650-f80f-43f6-b648-ee3f2d0c2dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05ec7e8b5ae04b2d84323ececfbf5acc)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:04:05.382126 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6353f650-f80f-43f6-b648-ee3f2d0c2dd5 admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:04:05.387148 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ad3dd-c3ff-4547-acd3-74e4d852df91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:04:05.388767 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ad3dd-c3ff-4547-acd3-74e4d852df91 service placement] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:04:05.400340 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1342/2663] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:04:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:04:05.421269 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:04:05.436775 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-6353f650-f80f-43f6-b648-ee3f2d0c2dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05ec7e8b5ae04b2d84323ececfbf5acc'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:05.437424 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-6353f650-f80f-43f6-b648-ee3f2d0c2dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05ec7e8b5ae04b2d84323ececfbf5acc'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:05.449171 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-6353f650-f80f-43f6-b648-ee3f2d0c2dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05ec7e8b5ae04b2d84323ececfbf5acc'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:05.456302 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-6353f650-f80f-43f6-b648-ee3f2d0c2dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05ec7e8b5ae04b2d84323ececfbf5acc'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:05.456913 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-6353f650-f80f-43f6-b648-ee3f2d0c2dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05ec7e8b5ae04b2d84323ececfbf5acc'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:05.469634 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1322/2664] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 22:04:05 2022] DELETE /identity/v3/users/05ec7e8b5ae04b2d84323ececfbf5acc => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:04:05.647125 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ad3dd-c3ff-4547-acd3-74e4d852df91 service placement] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:04:05.659845 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ad3dd-c3ff-4547-acd3-74e4d852df91 service placement] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:04:05.703934 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=100 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:04:05.722258 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0644664-3c1f-48a3-ba81-8563979032ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:04:05.722889 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0644664-3c1f-48a3-ba81-8563979032ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:04:05.723183 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0644664-3c1f-48a3-ba81-8563979032ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:04:05.723442 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0644664-3c1f-48a3-ba81-8563979032ec admin admin] PATH_INFO: `/v3/projects/15649df008a249a39e6d71c40a020954` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:04:05.724584 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0644664-3c1f-48a3-ba81-8563979032ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=15649df008a249a39e6d71c40a020954)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:04:05.726482 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0644664-3c1f-48a3-ba81-8563979032ec admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:04:05.776137 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c0644664-3c1f-48a3-ba81-8563979032ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15649df008a249a39e6d71c40a020954', 'request_id': 'req-c0644664-3c1f-48a3-ba81-8563979032ec'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:05.782079 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c0644664-3c1f-48a3-ba81-8563979032ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15649df008a249a39e6d71c40a020954', 'request_id': 'req-c0644664-3c1f-48a3-ba81-8563979032ec'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:05.789080 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c0644664-3c1f-48a3-ba81-8563979032ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15649df008a249a39e6d71c40a020954', 'request_id': 'req-c0644664-3c1f-48a3-ba81-8563979032ec'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:05.799512 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c0644664-3c1f-48a3-ba81-8563979032ec admin admin] The token cache is being invalidate because project 15649df008a249a39e6d71c40a020954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 22:04:05.799848 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c0644664-3c1f-48a3-ba81-8563979032ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:05.800474 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-c0644664-3c1f-48a3-ba81-8563979032ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:05.801948 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1343/2665] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 22:04:05 2022] DELETE /identity/v3/projects/15649df008a249a39e6d71c40a020954 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:04:05.806147 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353f650-f80f-43f6-b648-ee3f2d0c2dd5 admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:04:05.807548 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-6353f650-f80f-43f6-b648-ee3f2d0c2dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:04:05.820401 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353f650-f80f-43f6-b648-ee3f2d0c2dd5 admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:04:05.877697 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05c182-cbb9-49ab-9cba-9cf96ff2648a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:04:05.878239 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05c182-cbb9-49ab-9cba-9cf96ff2648a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:04:05.878517 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05c182-cbb9-49ab-9cba-9cf96ff2648a admin admin] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:04:05.878790 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05c182-cbb9-49ab-9cba-9cf96ff2648a admin admin] PATH_INFO: `/v3/users/9fd03c04e4194579be1822ba5f0d2320` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:04:05.880192 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05c182-cbb9-49ab-9cba-9cf96ff2648a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fd03c04e4194579be1822ba5f0d2320)` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:04:05.882089 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05c182-cbb9-49ab-9cba-9cf96ff2648a admin admin] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:04:05.926803 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0d05c182-cbb9-49ab-9cba-9cf96ff2648a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fd03c04e4194579be1822ba5f0d2320'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:05.927453 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0d05c182-cbb9-49ab-9cba-9cf96ff2648a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fd03c04e4194579be1822ba5f0d2320'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:05.939196 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0d05c182-cbb9-49ab-9cba-9cf96ff2648a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fd03c04e4194579be1822ba5f0d2320'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:05.946072 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0d05c182-cbb9-49ab-9cba-9cf96ff2648a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fd03c04e4194579be1822ba5f0d2320'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:05.947316 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.notifications [None req-0d05c182-cbb9-49ab-9cba-9cf96ff2648a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fd03c04e4194579be1822ba5f0d2320'} {{(pid=81305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:05.959241 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1323/2666] 10.222.0.13 () {58 vars in 1224 bytes} [Fri Oct 21 22:04:05 2022] DELETE /identity/v3/users/9fd03c04e4194579be1822ba5f0d2320 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:04:06.127428 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0644664-3c1f-48a3-ba81-8563979032ec admin admin] Authenticating user token {{(pid=81306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:04:06.128917 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.fernet_utils [None req-c0644664-3c1f-48a3-ba81-8563979032ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:04:06.148074 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0644664-3c1f-48a3-ba81-8563979032ec admin admin] Validating token access rules against request {{(pid=81306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:04:06.204496 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99bc67-b488-4332-bc7a-7bec75cede72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a2a3bc91ec4aebbb1e136167a1d87f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64a13af16ef54c01b2a28b4530062691', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:04:06.205114 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99bc67-b488-4332-bc7a-7bec75cede72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:04:06.205388 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99bc67-b488-4332-bc7a-7bec75cede72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:04:06.205655 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99bc67-b488-4332-bc7a-7bec75cede72 admin admin] PATH_INFO: `/v3/projects/cf30f933d49d427495042cc70ef2ac7a` {{(pid=81306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:04:06.206737 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99bc67-b488-4332-bc7a-7bec75cede72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf30f933d49d427495042cc70ef2ac7a)` {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:04:06.208727 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99bc67-b488-4332-bc7a-7bec75cede72 admin admin] RBAC: Authorization granted {{(pid=81306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:04:06.252032 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7a99bc67-b488-4332-bc7a-7bec75cede72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf30f933d49d427495042cc70ef2ac7a', 'request_id': 'req-7a99bc67-b488-4332-bc7a-7bec75cede72'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:06.258355 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7a99bc67-b488-4332-bc7a-7bec75cede72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf30f933d49d427495042cc70ef2ac7a', 'request_id': 'req-7a99bc67-b488-4332-bc7a-7bec75cede72'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:06.265955 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7a99bc67-b488-4332-bc7a-7bec75cede72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf30f933d49d427495042cc70ef2ac7a', 'request_id': 'req-7a99bc67-b488-4332-bc7a-7bec75cede72'} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:06.273104 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG dbcounter [-] [81306] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:04:06.277611 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7a99bc67-b488-4332-bc7a-7bec75cede72 admin admin] The token cache is being invalidate because project cf30f933d49d427495042cc70ef2ac7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 22:04:06.277947 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7a99bc67-b488-4332-bc7a-7bec75cede72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:06.278884 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: DEBUG keystone.notifications [None req-7a99bc67-b488-4332-bc7a-7bec75cede72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 22:04:06.280921 dv-cinder-861566-4-iscsi devstack@keystone.service[81306]: [pid: 81306|app: 0|req: 1344/2667] 10.222.0.13 () {58 vars in 1233 bytes} [Fri Oct 21 22:04:06 2022] DELETE /identity/v3/projects/cf30f933d49d427495042cc70ef2ac7a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 22:04:15.955837 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=71,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 22:06:31.005930 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05c182-cbb9-49ab-9cba-9cf96ff2648a admin admin] Authenticating user token {{(pid=81305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 22:06:31.007337 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-0d05c182-cbb9-49ab-9cba-9cf96ff2648a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:06:31.076164 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05c182-cbb9-49ab-9cba-9cf96ff2648a admin admin] Validating token access rules against request {{(pid=81305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 22:06:31.133234 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0177dbf-c02e-4247-81c2-41203e2f0a2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f2b5b95e8450ea6867f6868b9e810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2c33250ef84d79a4a4742fb4252156', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 22:06:31.133843 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0177dbf-c02e-4247-81c2-41203e2f0a2e service placement] REQUEST_METHOD: `GET` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 22:06:31.134152 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0177dbf-c02e-4247-81c2-41203e2f0a2e service placement] SCRIPT_NAME: `/identity` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 22:06:31.134424 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0177dbf-c02e-4247-81c2-41203e2f0a2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 22:06:31.136212 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.fernet_utils [None req-a0177dbf-c02e-4247-81c2-41203e2f0a2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 22:06:31.147301 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0177dbf-c02e-4247-81c2-41203e2f0a2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 22:06:31.148906 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0177dbf-c02e-4247-81c2-41203e2f0a2e service placement] RBAC: Authorization granted {{(pid=81305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 22:06:31.159306 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: [pid: 81305|app: 0|req: 1324/2668] 10.222.0.13 () {60 vars in 1365 bytes} [Fri Oct 21 22:06:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 22:06:41.155050 dv-cinder-861566-4-iscsi devstack@keystone.service[81305]: DEBUG dbcounter [-] [81305] Writing DB stats keystone:SELECT=58 {{(pid=81305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}